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PREFACE 


This book is the result of the writer’s conviction that 
the progress of research might be accelerated if those 
who engage in it possessed a little more training in its 
technique. Such training as graduate students get in 
research technique is largely of an incidental nature, 
except in the case of a few who take courses in statis- 
tical or historical method. The writer believes that 
much could be done in a more direct way as a prepara- 
tion for later individual work. 

The author’s hope is that this book may be used as a 
textbook in a regular course in the senior or graduate 
year, in preparation for entrance into graduate study 
and research. Allowing for seventeen effective weeks 
in the semester, the course would deal with one chapter 
a week, together with extensive supplementary reading 
and study in the library. In this way each student 
should get acquainted with the leading authorities on 
each type of research and the principles involved in it. 
In addition to this study of the fundamentals by all 
students in the class, the author suggests that each 
student might be expected to do two other pieces of 
work of an individual character. He might be expected 
to prepare a comprehensive paper based on very ex- 
tensive reading about the particular type of research 
in which he expects to engage, and also to do an actual 
piece of research employing that technique. If the 
student is a graduate, this piece of research might 
be the beginning of the thesis, or it might be merely 
a preparation for it. 

The material included here has been assembled by the 
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following processes: First, a complete difficulty analy- 
sis was made by having graduate students report those 
difficulties which they had encountered, and by having 
professors report those which they had experienced or 
observed in research workers, and by examining about 
one hundred theses for their shortcomings, defects, and 
implied difficulties. Second, these difficulties were classi- 
field into types and formulated into questions beginning 
with the word “How”. There were 145 such “How” 
questions, each representing a research difficulty. 
Third, all methods that could possibly be found for 
solving each of these difficulties were collected. These 
methods were drawn from about one hundred graduate 
students, a few professors, numerous books and articles 
about research, and scores of published and unpub- 
lished research reports. Each method, when found, 
was written on a small card and given the number of 
the difficulty of which it was a solution. Fourth, the 
cards were classified and arranged according to content, 
and their contents written in textbook form, with a 
goodly mixture of the author’s own thinking and evalu- 
ation of the methods presented. If the recommen- 
dations here made are not all sound, they will at least 
raise issues and stimulate thought. 

Special acknowledgements are due to Dr. W. W. 
Charters for the technique used in assembling the 
material for the book, to Dr. W. S. Ford, for supply- 
ing material for the chapter on Survey Technique, to 
the various authors of books, articles, and theses from 
which suggestions have been drawn, and to the many 
graduate students who supplied their lists of difficul- 
ties and methods. 

C. C. CRAWFORD. 
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IN EDUCATION 


Cra LER eT 
SELECTING PROBLEMS 


Two tasks confront the person who would begin edu- 
cational research. He must, first, discover problems 
that need solving, and, second, he must judge the rela- 
tive values of those he discovers. Each of these tasks 
will be discussed in this chapter. 


I. Finding Problems 


If every person who wished to engage in research 
could come to his professor or director with his problem 
clearly formulated, and his plans for attacking it 
already made for approval or revision, this would be 
ideal. More often, however, the candidate for research 
is faced more urgently by the need of doing something 
than by any definite problem that he wishes to solve. 
The search for a problem or the discovery of an issue 
which needs investigating can be made more productive 
by employing refined or systematic procedures. Blind 
searching is more desirable in this case than in the case 
of searching for objects of a material nature. The fol- 
lowing paragraphs are intended to assist in the process 
of discovering problems that lend themselves to scien- 
tific investigation: 
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1. Deriving a problem from a vocational choice. 
The person who has selected as a life work the teaching 
of English, the administering of a high school, or the 
handling of an adjustment room, is already in an excel- 
lent position to select a thesis problem. On the other 
hand, one who does not know whether he will teach 
history or mathematics is sometimes not ready to make 
the decision on a thesis problem. A piece of research 
has a double relationship with one’s vocation: (1) It 
will be a better piece of research if it fills a vital pro- 
fessional need. (2) It will contribute more to future 
professional growth and development if it is in the 
general field that is to be followed as a life work. 

2. Specializing in the field to be followed. There is 
actually such a thing as being too ignorant to ask in- 
telligent questions. The person who knows nothing 
about education will likewise find difficulty in discover- 
ing the vital issues that demand further study. Cer- 
tainly no one should attempt to explore the unknown 
and uncharted areas in education until he has acquired 
a reasonable familiarity with what is the common 
knowledge of well-informed educational workers. Occa- 
sionally this is done, with the result comparable to dis- 
covering the “new” principle of the bow and arrow in 
this modern age of machine guns and heavy artillery. 
Anyone who hopes to engage in research, whether for 
an advanced degree or not, should, in justice to himself, 
acquire a reasonably thorough foundation in the ordi- 
nary subject matter of the field in which he ultimately 
hopes to do his work. Such a foundation makes it 
easier to perceive the issues and to judge the import- 
ance of problems that present themselves. 

3. Analyzing a field or topic into its parts. Almost 
any educational topic, if reduced to its elements, would 
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break up into hundreds of excellent research problems. 
The beginner is usually acquainted with the major top- 
ics, such as supervised study, project method, socialized 
recitations, etc., and not uncommonly attempts such a 
topic instead of a problem. These topics are unsatis- 
factory because they are in too large units to be at- 
tacked as such. A little analysis has not only the 
advantage of reducing the size of the unit but also that 
of enormously increasing the number of units or parts 
which may be followed up as research problems. It 
has been the writer’s experience that most thesis prob- 
lems were arrived at by subdividing much larger topics 
into their parts rather than by the opposite plan. 

4, Extending term papers. It is customary in most 
advanced courses and in graduate seminars for each 
student to be asked to prepare a long paper on some 
subject connected with the course. While these papers 
are usually not of a research nature, they are neverthe- 
less an excellent preparation for a thesis in that they 
are exploratory and help to discover the problems that 
are involved in the field. A good policy for an advanced 
student who is about ready to begin research is to 
select several topics in which he is somewhat interested 
and to write a term paper on each in a different course; 
then to narrow one of them down to a thesis problem 
to be attacked by scientific research methods. 

5. Providing research contacts. It is fairly safe to 
judge a man by the company he keeps. Particularly is 
this true in research. A person who is interested in 
research will do well to enroll in courses with kindred 
souls and under professors who have a spark of the 
research fire in their hearts. There is in the usual col- 
lege faculty at least one man who is especially inter- 
ested in research and scholarly production. He may not 
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be the best teacher nor the most interesting person- 
ity on the campus, but his interest in research is strong 
and his power to inspire and to direct scientific work is 
great. The would-be research worker should find such 
a man and study under his direction. This usually 
involves enrolling in a graduate seminar, a veritable 
nursery for research workers. Such definite and for- 
mal contacts with persons interested in research can 
not help contributing greatly both to the discovery and 
the solution of research problems. 

6. Meditating. One way to discover research prob- 
lems, but a rather poor way as a rule, is by simply 
meditating. To dwell on the need or desire for a prob- 
lem long enough is likely to cause one to arise in the 
mind. Some active or systematic plan of attack, or 
some purposeful search for a problem, is much more 
likely to produce results. The most fruitful results of 
meditation are derived when it is employed in evaluat- 
ing problems after several possibilities have been defin- 
itely formulated. An active search for several prob- 
lems, followed by a period of “soaking,” or meditation, 
is to be encouraged. 

7. Analyzing popular superstitions. The history of 
science has.been a long succession of blasting of popu- 
lar superstitions. Copernicus discovered that the earth 
moves instead of the sun; Columbus demonstrated that 
the earth is round and not flat. Yet for years after 
these discoveries the masses clung to the older beliefs. 
Similarly, in our own day we find scores of popular 
superstitions about biology, psychology, sociology, edu- 
cation, etc., which are still cherished by the multitudes. 
The popular mind refuses to believe that dogs do not 
reason, or that study does not produce a general mental 
discipline. Any one of these issues, which may be 
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thought of as skirmishes between the scientist and the 
layman, would furnish an excellent thesis problem. If 
the masses are right, we should know it; if they are 
wrong, the more facts that are made available, the 
more likely are they finally to see the light and accept 
the scientific view. 

8. Particularizing feelings of discontent. There is 
probably no one who is entirely satisfied in every way. 
Even the best teacher or administrator is likely to be 
unhappy about something. The trouble with most peo- 
ple, however, is that they have not taken the trouble 
to sit down and decide exactly what they are displeased 
about. The lower animals are said to be unable to 
distinguish consciously the difference between the 
promptings of such impulses as hunger, thirst, cold, 
sex, and pain, and simply become so restless under one 
of these stimuli that they stir around until they happen 
upon suitable sources of relief or satisfaction. In 
contrast with this blind and aimless method of remedy- 
ing our troubles, scientifically trained research workers 
think the situation over or make a definite analysis of 
it until the exact source of irritation or annoyance is 
located, and then set up the necessary procedure to cor- 
rect it. Each cause of discontent is a potential thesis 
problem. 

9. Making an analysis of difficulties or deficiencies. 
“They that are whole have no need of the physician, 
but they that are sick.” The greatest need of research 
is in those phases of endeavor which are falling short 
of the mark. A principal who wishes to encourage 
research in his school has only to note where the school 
is weak, faulty, or deficient to find a multitude of re- 
search problems. The classroom teacher needs only to 
look over the red marks she hag put on a set of papers 
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to discover a host of research problems dealing with the 
learning process and methods of studying employed by 
her pupils. Every difficulty or deficiency is a potential 
research problem. 

10. Observing educaticnal practice. To watch a 
teacher or student at work for an hour should raise 
in the mind a number of problems for intensive study. 
The observation will bring to mind several unsatis- 
factory elements, several elements of unusual merit, 
or several matters that call for further interpretation. 
A few days devoted to observation and visitation of 
actual classrooms should raise in the mind such a 
number of questions that the difficulty would no longer 
be in finding a problem, but in deciding which of several 
problems to follow up. If this plan were employed 
more frequently by research workers, there would be 
fewer purely abstract investigations and more genuine 
solutions of the vital issues now troubling educational 
practitioners. : 

11. Listening with a critical attitude. What people 
say is usually less carefully thought out and less 
thoroughly considered than what they write. Even 
scientists of some repute who would not publish mere 
opinions dogmatically will sometimes speak them orally 
with an air of absolute finality. The person of less 
training is even more prone to say what is on his 
mind without taking the trouble to examine its truth- 
fulness. The result is that lectures, round-table dis- 
cussions, and class recitations afford an innumerable 
supply of possible thesis problems, because almost 
every sentence that is uttered is likely to contain some- 
thing that is unproven and therefore in need of further 
investigation. Listening to an ordinary argument 
should suggest to a scientific mind very quickly what 
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essential facts are needed by the contestants, or what 
investigations would be required to settle the issue. 
Most lectures and class discussions involve an element 
of argumentation, and every subject that lends itself 
to argumentation is at least a possible candidate for 
scientific investigation. 

12. Reading research reparts. Research begets re- 
search. One man’s investigation inspires and suggests 
scientific work on the part of others. To read what 
another person has done on a given problem helps to 
see the issues and to decide upon a phase of it for one’s 
own study. To read several theses or published articles 
of a scientific nature will almost certainly help in 
making a choice of a problem. Most graduate schools 
have, in addition to the theses produced by their own 
students, more or less complete files of the Columbia 
University doctors’ dissertations. A few days spent 
in reviewing these would be worth the time of any 
graduate student who is getting ready for his own 
thesis study. Not only does the reading of such works 
contribute greatly in making clear the nature of 
scientific work, but also in suggesting the further work 
that needs to be done on each of the problems that 
have been investigated. To take up any ordinary 
thesis problem and apply to it the remaining prcce- 
dures which the previous investigator overlooked or 
omitted would constitute on entirely satisfactory thesis 
undertaking. 

13. Reading current periodicals. There are two 
reasons why educational periodicals are especially 
helpful in the selection of research problems: (1) 
They are of a research nature to a greater extent than 
books. Books, as a rule, present the thoughts of their 
authors, whereas the periodicals present the facts 
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derived from investigations. For this reason a person 
hunting a thesis topic will profit more by going through 
periodicals than by going through books. (2) Period- 
icals are more recent than books. Books are often out 
of date by the time they are published, and hence do 
not suggest the newest things in educational science. 
Periodicals, however, present what is on the very 
“firing line’ of educational activity. Since research 
is by nature something of a pioneer or frontier work, 
it is especially important to move up to the frontier 
to begin, by reading what is going on at the time. 
Without reading the journals a student cannot be sure 
that a proposed problem has not already been solved 
by another investigator, or outgrown by the march 
of events. 

14. Reading textbooks critically. The usual text- 
book in education is only partially based on scientific 
data. Research has not gone far enough to provide 
data for guidance on matters of educational practice. 
But textbook writers cannot wait. They are obliged 
to provide something for the training of the new gen- 
eration of educational workers, and in the absence of 
science they are forced to resort to common sense or 
empirical advice. To read a textbook on methods, ad- 
ministration, or educational psychology with this point 
in mind should suggest a host of problems for inves- 
tigation. Each chapter title might be considered a 
thesis topic and almost any paragraph or sentence in 
the chapter might be made into a thesis problem. The 
chief essential is that the reading be done not from 
the viewpoint of assimilation but of investigation or 
critical searching for points which are stated without 
scientific verification. 

15. Using questions and exercises in textbooks. 
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Most textbooks have some kind of “questions for fur- 
ther study” or “topics for investigations,” etc., at the 
ends of chapters, which are useful as a source of re- 
search problems. 

16. Consulting bibliographies of scientific materials. 
To look over a list of titles of theses accepted by any 
graduate school, or to note the titles of research ar- 
ticles published during a given period, in an excellent 
way to discover problems for investigation. The 
United States Bureau of Education occasionally pub- 
lishes in bulletin form bibliographies of research under 
way or completed. The Phi Delta Kappa educational 
fraternity publishes in its official organ the titles of 
masters’ and doctors’ theses accepted during the cur- 
rent year at the various institutions which have fra- 
ternity chapters. The Educational Research Associa- 
tion has a policy of preparing a summary of biblio- 
graphy of educational investigations at intervals. All 
such lists of research endeavors have splendid possi- 
bilities in suggesting problems for scientific study. The 
mere fact that some one else has undertaken a given 
study does not eliminate it from the list of possible 
thesis subjects. It simply indicates that someone has 
opened up the field for others who are to come later 
and cultivate it more thoroughly. 

17. Deriving problems: from procedures. The pos- 
session of a certain tool stimulates the search for 
something upon which to use it. A boy who has a 
hammer and saw will think of something to make 
with them, just as a boy who wishes to make a bird- 
house will search for materials with which to make it. 
Many research problems are chosen in this way. A 
person who has an excellent training in statistics will 
consequently find a problem that will utilize the avail- 
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able tool. Another who has had training in methods 
of historical research will turn his energies upon a 
historical problem. It is not at all amiss to examine 
the various types of research procedure which are 
available and to select a thesis problem that grows out 
of one of these types. Procedures naturally suggest 
problems, and sometime they do so to a greater extent 
than problems suggest procedures. 

18. Assembling several problems from which to 
choose. To choose a problem from a large number of 
possibilities is preferable to taking the first one that 
presents itself. A good plan is to provide a sheet in 
the notebook headed ‘‘Research Problems” and to write 
down one whenever it comes to mind. After a period 
of time the collection will be comparatively large and 
the selection of the best one will be a matter of small 
difficulty. Unless such a collection of possibilities is 
made, some of the best problems that have occurred 
to the mind will have been forgotten by the time of 
the final decision, and will be completely overlooked. 
It is difficult to judge anything except by comparison, 
and the greater the number of other items with which 
to compare it, the more nearly perfect is the judg- 
ment made. 

19. Reading intensively on the problem tentatively 
selected. After a problem has been discovered which 
seems to meet the requirements for a thesis, the next 
step is to confirm or verify the selection by further 
study upon it. It is necessary to read and think about 
it enough to find what factors are included in it, what 
difficulties are presented, and what special conditions 
or circumstances will be required for its completion. 
By reading what other investigators have done on the 
same problem, it is possible to judge whether the diffi- 
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culties can be overcome and whether it can be carried 
through to completion. It is better to make such a 
survey of the field at this stage than to rush ahead 
blindly and then have to give up the undertaking after 
weeks or months of effort have been expended. 

20. Cenferring with faculty members. Almost every 
professor in a graduate school has in mind one or more 
vital issues in his subject or field which need inves- 
tigation. It is not uncommon for faculty members 
to have prepared lists of research problems that center 
around some special interest and which, taken as a 
whole, constitute a rather complete and exhaustive 
analysis of that one field. Since research is by nature 
somewhat opportunistic, it is often better to take a 
problem of this type, which is only a part of a much 
larger problem in which the professor is interested, 
and to work on it in company with numbers of other 
graduate students, than to choose one of an isolated 
type and work apart from the others. There is yet 
another reason for conferring with the professor in 
choosing a problem. The beginner usually lacks per- 
spective and foresight, and is likely to judge poorly the 
merits of a problem. As a corrective for this fact 
it is necessary to consult some one who can see more 
clearly the issues involved before finally making a 
selection. 

21. Letting the first problem grow. There is seldom 
any difficulty in finding or selecting a research problem 
after the first investigation is completed. A master’s 
thesis is almost certain to suggest a dozen or more 
suitable problems for a doctor’s dissertation. The 
more research a person does, the more problems he 
sees that call for investigation. Getting started on the 
first problem may be somewhai difficult, but this should 
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not cause discouragement or deter one’s entrance upon 
a research career, because it is a difficulty that does 
not often present itself again. 


II. Standards for Judging Problems 


The making of a final choice after several possible 
problems have been found is a matter of no small 
difficulty or importance. Several matters have to be 
considered, and the great danger lies in basing a deci- 
sion on only a few of the many elements involved. 
There is need for something in the way of a score-card 
or set of standards for estimating the desirability of 
problems for research. The following paragraphs 
discuss some of the items which might well be con- 
sidered in such a situation: 

1. Scientific character. Dr. Judd has called atten- 
tion to the difference between studies which are pri- 
marily research and those which are primarily ser- 
vice.* There are many things which can be done that 
involve accumulation of data, tests, making statistical 
calculations, etc., and which nevertheless are not re- 
search because they are simply doing something with 
present knowledge. Giving tests and teaching classes 
are worth-while activities, but are not research because 
they do not involve discovery of new truths or con- 
tributing to educational science. The student should 
examine the strictly scientific character of the research 
problem, asking always the question, “What new 
knowledge will its solution add to the world’s present 
supply?” rather than “How much immediate service 
will it render in a particular educational situation?” 

2. Fundamental importance. Some researches lead 


*Address delivered at a meeting of the Phi Delta Kappa 
fraternity. 
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to real inventions; others are merely tinkering with 
inventions previously made. Some affect the major 
channel of the stream of human progress; others 
merely cause minor eddies in the stream. Some dis- 
coveries will affect a few individuals in rather im- 
portant ways and not touch the masses; others will 
actually change the daily behavior of millions of peo- 
ple. The importance of a scientific investigation may 
be measured in two principal ways: First, how much 
territory or how many people will be influenced by it? 
Second, how much will these people be influenced? To 
raise the level of the ocean one inch would be a more 
important achievement than to raise the level of a 
small pond a yard. 

3. Facilitation of other research. The solving of a 
problem- should pave the way for the solution of still 
other problems. It should suggest new problems and 
clear the atmosphere so that they may be examined 
with greater precision than before. It may result in 
a new tool or technique, or a new method of analyzing 
data that can be used in attacking numbers of other 
problems. This point may be illustrated by comparison 
with a street car system. Some problems are like 
inbound cars which connect with dozens of other cars 
in the central traffic district, while other problems 
are like outbound cars which simply go to the end 
of the line. | 

4. Unification of knowledge. Some research under- 
takings are valuable in themselves, while others are 
valuable as connecting links between the former type. 
For example, Gray’s “Summary of Reading Investiga- 
tions’’* is a scholarly piece of work whose major con- 
tribution lies in its being an inventory of other peoples’ 


*Elementary School Journal, XXVII, March, 1927, pp. 495-510. 
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work in this field. An investigation may be of great 
value simply by establishing a link between other 
studies or by correlating, coordinating, or unifying the 
results of previous research. 

5. Timeliness. “Strike while the iron is hot” is an 
old proverb whose application to our present discussion 
is important. It means that some problems are un- 
desirable for research because they have been out- 
grown or passed by in the march of educational pro- 
gress. Every year some young enthusiast proposes to 
do what has been done hundreds of times already by 
previous workers, not knowing that he could find in 
a few hours of library work a more complete answer 
to his question than he would achieve after months of 
labor. There are, of course, some probems which, 
although old and much-worked, need to be reopened 
and examined in the light of more recent developments. 
However, one who takes such a problem should do so 
consciously and not in ignorance of its status. There 
is also an opposite side to the factor of timeliness, in 
that some problems are premature. It is possible to 
get a problem for whose solution the world is not ready. 
The present supply of knowedge in certain fields is so 
limited, and the available tools or instruments are so 
erude, that it is necessary in such fields to undertake 
research of the “scaffolding” type, getting ready for 
more ambitious undertakings later. 

6. Definiteness. A very desirable quality in a re- 
search problem is that which enables the investigator 
to know when he is through. The writer has known 
graduate students who were wondering when their 
professors were going to tell them to stop, who were 
doing their work very much as day laborers, and for 
whom there was no definite end to the task. Such 
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is most unfortunate. If a problem cannot be so defi- 
nitely formulated at the outset that a good idea of the 
goal can be obtained, it had better never be attempted. 
Even Columbus had a goal when he set out on his 
epoch-making journey, and the average graduate stu- 
dent who attempts to explore unknown realms of know- 
ledge should have a pretty clear idea of the goal ahead 
of him. 

7. Unity. The common tendency of untrained re- 
search workers is to include too many things in the 
problem. They start out with a fear that there will 
not be enough material to fill the required number of 
pages, and, consequently, try to cover a field large 
enough for ten or a hundred theses. They struggle 
with this great variety of problems and facts in a 
superficial way for a period sufficient to have solved 
one of the problems well, and then either quit in dis- 
gust or dress up what they have and hope to get it 
accepted. If they succeed, they sigh in relief and 
rejoice that their research careers are ended. It is 
almost impossible to overemphasize the need of anal- 
yzing the problem down to a unit and then solving 
that one problem. This makes for small and minutely 
specialized theses, but the day is past when any other 
kind will have much value. Any single unit, if prop- 
erly attacked and persistently followed up, will grow 
to such proportions and gather such significance as 
to satisfy any normal cravings of the investigator for 
importance and respectability. A minutely specialized 
investigation is by no means a trivial or worthless one. 
On the other hand, the worthless studies are much 
more commonly those which undertake to cover the 
whole field and analyze several factors rather than to 
exhaust the possibilities with a single unit. 
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8. Feasibility. How to prevent crime is a very 
worth-while problem, but one which the average re- 
search worker would do well not to attempt, because 
of lack of access to data, and lack of opportunity for 
experimentation. A problem must be judged in the . 
light of the facilities available for attacking it. It 
must be selected for a particular person and a partic- 
ular situation. Before committing himself to a prob- 
lem, a student should draw up a detailed outline of the 
procedure required to solve it, and unless’ he can see 
through to the end, he should not attempt it. One 
caution should be observed at this point, however. 
A good problem should not be abandoned merely be- 
cause a plan of solution does not automatically present 
itself. Feasibility is achieved by making the necessary 
conditions as often as by finding them. There are ways 
of attacking a great many problems that are not 
obvious at first glance, and the true scientist will not 
abandon his problem on grounds of feasibility until 
he has demonstrated to his entire satisfaction that 
it cannot be solved. 

9. Practical value to society. All research should 
ultimately be practical or useful. Knowledge for its 
own sake may be permissible at times, in the sense 
that its practical utility at the time may not be visible, 
but the ultimate goal should be knowledge for the 
welfare of mankind. In judging a problem one ques- 
tion that should be asked is, “What contribution will 
it make to society?” or “How much will it improve 
social practice?’’ Not only should a problem have some 
potential worth to society, but it should have more 
worth than others which are discarded in its favor. 

10. Personal returns involved. Until the time comes 
when research is recognized as a public service and 


SELECTING PROBLEMS 25 


paid for just as is teaching or entertaining or healing 
the sick, the research worker will have to give con- 
siderable thought to the personal rewards he is to gain 
from his work. This does not mean, however, that 
he is to consider purely the financial profit that comes 
from the work, for there are crdinarily considerably 
greater rewards of other kinds than the financial. One 
reward to be considered is that of increased training. 
Not every problem will give equally valuable practice 
in specialized techniques which make for future pro- 
ductiveness. A second form of personal reward is 
professional growth and development. It is well to 
choose a problem with a forward look, thinking of a 
whole career rather than the immediate necessity of 
meeting a degree requirement. To fail to consider the 
relation of a research undertaking to one’s future pro- 
fessional career is comparable to studying for an ex- 
amination with no intention of remembering anything 
after it has been passed. 

11. Freedom from prejudice. If the main point of 
a certain problem is to prove something, it should be 
abandoned. If it is one upon which the investigator 
has a strong hobby, he will be unable to pursue it 
scientifically. A mother is unable to see the short- 
comings of her own children, however obvious they 
may be to the neighbors, and an investigator is unable 
to think clearly and draw conclusions wisely about the 
project or pet scheme that is dear to his heart. Before 
starting on a problem, it is well to ask, “Is this a 
problem upon which I am open-minded and, about 
which I can fearlessly face the truth?” | 

12. Previous preparation required. A good problem 
for one person may be absolutely unsuited to a person 
of different experience and training. A person who 
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has not had statistical training should by no means 
undertake statistical problems, and lack of historical 
training would bar any attempt at preparation of a 
thesis of the historical type. Persons who have had 
teaching experience are able to take up certain prob- 
lems which inexperienced persons could not hope to 
solve successfully. A thesis problem should be 
selected only after a careful analysis of the amount 
and kind of previous preparation it presupposes. 

13. Interest. Other things being equal, that prob- 
lem is best which holds the greatest interest for the 
investigator. There are several ways to fulfill this 
requirement in selecting a thesis problem: (1) Assume 
a large share of the responsibility for selection of the 
problem instead of putting it upon the faculty direc- 
tor. (2) Read extensively about the problem before 
finally selecting it, in order to see what it involves. 
Research is like marriage in that thorough acquaint- 
ance beforehand may save many regrets later. (3) 
Start with the main dominant interests and work to- 
ward the thesis problem. For example, an English 
teacher would begin the search for a thesis problem 
within the field of English. (4) Read research reports 
and select problems similar to those which prove most 
interesting. (5) Choose one arbitrarily and get in- 
terested later. A research problem does not prove 
interesting in the same way as a motion picture show. 
Its interest grows out of constant association and 
mental application upon it. It is possible to get in- 
terested in almost any problem. (6) Derive interest 
from the procedure employed in addition to the prob- 
lem itself. It is possible to.make almost any problem 
interesting by being resourceful in setting up methods 
of attacking it. 
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EXERCISES AND QUESTIONS 


1. In what sense is it true that the genuine research worker 
has no difficulty in finding a problem? 


2. Why does a research director usually dislike to assign a 
problem for a thesis? 


3. Using the present chapter as a source of suggestions, list 
as many research problems as you can think of. 


4. Apply to the list sugested in 8 above the standards out- 
lined in the second half of this chapter. 


5. Describe your state of mind when you first began to con- 
sider the choice of a research problem. 


6. Arrange in the order of their value the methods of finding 
problems mentioned in the first half of the chapter. 


7. Do the same for the standards described in the second half. 


8. If you have done a piece of research or selectd a problem, 
tell how you made the selection. 


9. Read in other books and articles on research and prepare 
a short paper or outline on “Selecting a Research 
Problem.” 
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CTTAPTER« TI 
EXPERIMENTAL TECHNIQUE 


1. Nature of the experimental method. The experi- 
mental method has in recent decades come to be the 
backbone of science. Originating as a form of logic, 
or as a method of thinking, rather than as a definite 
and systematic procedure, it has now reached the stage 
where it is recognized as probably the most essential 
instrument available in the quest for truth. In its 
bare essentials, it consists simply of providing a situa- 
tion where phenomena can be repeated and controlled, 
and then varying one element in the situation to see 
what variation in the result is thereby produced. This 
means the isolation of a single variable and the deter- 
mination of its results when the other variables or 
elements in the situation remain constant. For ex- 
ample, if the growth of a plant is dependent upon sev- 
eral variables, as light, heat, moisture, soil, and culti- 
vation, and the problem is to determine the part played 
by cultivation, the experimental method would be to 
measure growth under different forms of cultivation, 
being careful to avoid changes in amount or character 
of light, heat, moisture, or soil. If two of these ele- 
ments are permitted to vary at the same time, it is 
then impossible to know which one is responsible for 
the variation in the result produced. 

2. “Practical” versus scientific experiments. There 
is a great difference between experiments and exper- 
ience. An experiment, in the scientific sense, involves 
a definite and conscious technique, with special atten- 
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tion to the measurement of results, control of the 
conditions, and the drawing of accurate and precise 
conclusions. It is far from a mere haphazard or trial- 
and-error procedure. A lower animal enclosed in a cage 
may “experiment” with methods of releasing himself, 
but not in the scientific sense. Likewise, many edu- 
cational practitioners “experiment” with new methods 
of teaching or administration without taking the 
trouble to control conditions or measure results in 
any but the very crudest fashion. Such “experiment- 
ing’”’ probably has much value, particularly as a con- 
tribution to the skill or art of the persons involved, 
but it usually adds little or nothing to educational 
science because the conclusions drawn are doubtful and 
the conditions are so poorly controlled that they can- 
not be repeated by others in exactly the same fashion. 
There is great need for more scientific experimenting 
such as is being done in the laboratory schools in large 
universities, in the place of the present “practical” 
or ‘“rough-and-ready” experimenting now done by 
teachers and administrators who have had little scien- 
tific training. 

3. Typical conditions. The conclusions from an ex- 
periment do not necessarily apply outside of the situa- 
tion in which the experiment was performed. For 
example, an experiment on white rats does not neces- 
sarily hold true for human beings, nor does an ex- 
periment with adults necessarily apply to children. 
An experiment may be such as to create an entirely 
unusual, or even abnormal, situation in the classroom, 
raising interest to fever heat and producing spectac- 
ular results which would not appear in the normal 
classroom situation. Thorndike’s experiments with 
animal learning have been criticised slightly because 
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he starved the animals to provide a motive for escap- 
ing from their confinement, and thereby may have 
produced an abnormal organic condition in them which 
affected their learning. Many educational and psychol- 
ogical experiments have been of doubtful validity 
because they were carried out under special conditions 
in the laboratory that were not typical of the class- 
room where the conclusions were expected to apply. 
4, Subjects. The persons, animals etc., experi- 
mented upon are called subjects. An experiment is 
no better than the subjects involved. If they are 
improperly selected, the conclusions are vitiated. If 
they have a bias and have an opportunity to express 
it in their reactions, the experiment becomes ques- 
tionable. They must be representative of the groups 
to which the experimental conclusions are to be applied 
and must be appropriate to the experimental factors 
or variables being tested. If the desire is to discover 
laws or generalizations about reading in the third 
grade, college students are not suitable subjects for 
the experiments. Nor is a class of honor students 
suitable for an experiment if the conclusions are to 
be applied to children of all degrees of ability. The 
number of subjects is also very important. It is better 
to use a very few subjects and control conditions thor- 
oughly than to do superficial work with a large num- 
ber. The ideal, of course, is to have careful work 
with a large number, but a compromise is usually re- 
quired. As a rough working rule, it is fairly well 
agreed that for an experiment on human mental re- 
actions, fewer than thirty subjects is likely to yield 
questionable results. On the other hand, having over 
one hundred subjects is likely to increase the labor 
at a rate disproportionate to the educational returns. 
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If an experimental program is to involve over one hun- 
dred subjects, it is well to ask whether it would not 
be better to do two or more allied and mutually sup- 
plementary experiments with fewer subjects involved 
in each. This, however, is a relative matter, and 
depends somewhat upon the nature of the problem, 
If the element to be measured is of a very delicate 
nature, it may be necessary to get very large numbers 
of subjects in order to get reliable results. 

5. Duration. It is impossible to say definitely how 
long an experiment should continue. Much depends on 
the particular case. As a general rule, the tendency 
is to err on the side of too short a time rather than 
too long. A two-week experiment to determine the 
advantages of married life would hardly be valid, nor 
would a six-week experiment with a program of homo- 
geneous grouping. Some of the results of an experi- 
ment may not mature until even years afterward, in 
which case a short-term procedure would overlook im- 
portant results. It is much more difficult to judge a 
100-yard dash than a mile run, because there is less 
time for the differences which exist to manifest them- 
selves. If an experiment is allowed to continue long 
enough to produce a drastic change, or a big sampling 
of its results, minor errors will not be so serious. On 
the other hand, a long-term experiment would fail to 
discover certain things which are shown by a short- 
term one. For example, a one-hundred yard dash is in 
large part a measure of reaction time, or speed of 
starting, while the mile run is more a measure of fun- 
damental body energy. The duration of the experi- 
ment will depend on what is being measured. Some- 
times it is well to conduct a long-term experiment and 
to observe results at various intervals during the 


EXPERIMENTAL TECHNIQUE 33 


period, thus combining one long-term experiment with 
a series of short-term ones. 

6. Precision. An experiment may suffer from being 
either too precise or not precise enough. The totally 
untrained experimenter usually lacks precision, being 
satisfied with gross results only. He may even be con- 
tent with qualitative results and not bother about 
quantities at all. The beginner who has learned some- 
thing about thé importance of precise methods, but 
who has not had enough practice to develop judgment 
in such matters often “stands up so straight that he 
leans backward,” burdening his procedure with so 
many technicalities that the experiment breaks down 
from its own complexity. Sacrifices must be made 
somewhere in practically every experiment. If an 
airplane carried all the equipment, instruments, land- 
ing gear, parachutes, life belts, rafts, etc. etc., that 
might possibly be useful in a trans-Atlantic flight it 
would have no place for gasoline. Likewise, simplicity 
is a great asset in an experiment, but it cannot always 
be obtained without sacrificing precision, or omitting 
special techniques that would add valuable but not 
indispensable information. This is not a plea nor a 
justification for superficial work, but merely a sugges- 
tion for the application of common sense. Distances 
between cities are measured in miles, but the size of 
automobile pistons is measured in more precise units. 
The ideal is to adjust the degree of precision to the 
nature and importance of the thing being studied. 

7. Individual differences. A perfectly controlled ex- 
periment on human subjects regarding mental be- 
havior is practically impossible. It is next to im- 
possible to be sure that such factors as_ interest, 
memory, intelligence, mental set, etc., are the same 
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for all subjects. In physics or chemistry the problem 
is much simpler. A piece of iron or a gram of nitric 
acid has the same properties and the same behavior 
as any other unit of the same substance. Conse- 
quently, any sampling may be considered truly repre- 
sentative of the class to which it belongs. But not so 
of human beings. No two are alike. What holds true 
for one may be only partially true for another, and. 
absolutely false for a third. Experiments with human 
beings are made exceedingly complicated by this fact. 
An experiment on one person is of little value because 
he may be quite unusual. It is necessary, therefore, 
to experiment with several persons and then get some 
kind of average of the various individual results, which 
may be considered the generalization for human beings 
as a whole. This fact makes the choice of the samp- 
lings very important, and also calls for the use of more 
or less specialized statistical procedures to bring the 
various individual results together into some unified 
coefficient which expresses the uniformity running 
through all the cases. 

8. Planning. The expression, “Fools rush in where 
angels fear to tread,” applies to experimental research 
quite appropriately. The real scientist is not in a 
hurry to get started. He takes plenty of time to 
arrange the situation and to gain access to needed 
materials and equipment. There are several types of 
preparation which should be made before actually 
starting the performance of an experiment: 

(1) Studying the rules of the game. Anyone un- 
dertaking experimental work should take time to ed- 
ucate himself with respect to its technique by reading 
articles and books about it. By far the best practical 
guide for this purpose now available, in the writer’s 
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estimation, is McCall’s How to Experiment in Education, 
which should be purchased and used as a manual by 
the research worker. 

(2) Formulating a hypothesis. It is best to spend 
considerable time in thinking the problem through to 
a working hypothesis and then to make the experiment 
a test or verification of that hypothesis. More energy 
and thought at this stage make unnecessary much 
superfluous effort in manipulating test scores and sta- 
tistics that do not contribute anything. 

(8) Making a written plan. Formulating the pro- 
posed steps in the procedure in writing is an excellent 
method of planning for an experiment. There are so 
many elements and considerations involved that any- 
things short of a written plan is likely to be frag- 
mentary and inadequate. 

(4) Getting criticism of the plan. One person is 
never as likely to discover all the flaws in a proposed 
procedure as two. When the second person happens 
to be an expert in research work, as is usually the case 
with a major professor or research director, a con- 
ference over the first draft of the plans is almost 
certain to result in valuable improvements. 

(5) Rehearsing the experiment with a small group. 
Writing and revising the plan involves a mental or 
imaginary rehearsal of the procedure, but it is usu- 
ally desirable, wherever feasible, to carry it out on a 
small scale or with a few subjects, and to revise again 
before going into it finally for serious results. Almost 
every experimental research report mentions some- 
thing in the nature of a defect or shortcoming that the 
experimenter discovered after it was too late. There 
will be plenty of these even after a preliminary re- 
hearsal. 
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9. One-group procedure. McCall’s How to Experiment 
in Education gives an excellent outline of procedure for 
three types of experiments: the one-group, the equiv- 
alent-groups, and the rotation experiments. For de- 
tails of these types, the reader should consult that 
book. The present and the two following paragraphs 
will only attempt to touch some of the high points of 
each. The one-group procedure involves a single group 
of subjects and the application to that group of two 
or more variables in succession, all other conditions 
or variables being kept constant during the entire 
period. For example, if the purpose were to deter- 


TABLE I 
PROCEDURE FOR A ONE-GROUP EXPERIMENT 


Period Variable 


(Note: The table is to be read as follows: During the first 
period the first variable is applied, and during the second 
period, the second variable is applied. The table would 
simply be extended in the case of three or more variables.) 


mine the effect upon pulse rate of jumping as com- 
pared with running, the group would be assembled 
when the pulse rate was normal and be given one of 
these forms of exercise, and the increase noted. Then, 
at another period, with pulse rate again normal, they 
would be given the other form of exercise and the 
increase noted and compared with that from the first 
form. The procedure for a one-group experiment is 
shown diagramatically in the Table I. The great weak- 
ness of the method is that in most educational activi- 
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ties the condition does not return to normal after the 
first period. There is usually a hang-over from the 
first to the second period which prevents a true com- 
parison. Moreover, in most forms of learning, the 
progress is at different rates at different portions of 
the learning curve. For example, if a typewriting 
class were taught by one method for one semester 
and by a different method the second semester, with 
increase in rate of typing used as the measure of 
efficiency, the first method wculd almost invariably 
be found superior, because it is natural for speed to 
increase more rapidly at first than in the later stages 
of practice. 

10. Parallel-group procedure. One way to overcome 
the difficulties noted in the case of the one-group pro- 
cedure is to use two groups of equal ability in every 
way, applying one variable to one while the other var- 
iable is being applied to the other. This places both 
groups in the same portion of the learning curve, and 
eliminates any hang-over from one variable to the 
other. The only serious difficulty lies in being abso- 
lutely certain that the two groups are exactly equal, 
or that the treatment of the two groups is equal in 
every way except in the two variables being compared. 
The various procedures in equating the groups are as 
follows: (1) To equate by chance, as by drawing lots. 
(2) To equate by intelligence, pairing subjects of equal 
scores against each other. (38) To equate by achieve- 
ment tests, pairing students as before. (4) To equate 
on the basis of the initial test in the traits or abilities 
being studied. (5) To equate by the accomplishment 
quotient, or ratio between achievement and ability to 
achieve. (6) To equate on a multitude of factors by 
weighting the scores in each and putting all together 
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into a composite score. It is important that the par- 
allel groups have not only the same average scores 
but also the same range of distribution of scores. For 
example, a group of 100 subjects should not be divided 
by placing the middle fifty in one group and the upper 
and lower twenty-fives in the parallel groups, even 
though this procedure would yield the same averages 
omit for the two groups. 


TABLE II 
PROCEDURE FOR PARALLEL-GROUP EXPERIMENT 


(Note: The table is to be read as follows: Variable 1 would 
be applied to Group A and variable 2 to Group B. For 
three or more groups the table would simply be extended.) 


11. Rotation procedure. The rotation procedure has 
most of the advantages and few of the disadvantages 
of both the one-group and the parallel-group pro- 
cedures. In fact, it is simply a combination of these 
two procedures. It may be thought of as two parallel- 
group experiments in which the variable applied to 
group A one time is applied to group B the next. By 
this means each variable is applied to each group, 
thus eliminating the problem of equality of groups 
as well as the problem of different rates of progress 
in different portions of the learning curve. It does 
not absolutely eliminate the possibility of more hang- 
over from the first period to the second for one var- 
iable than for the other, but it reduces the liklihood 
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of such a thing to a minimum. A wider adoption of 
this procedure is very much needed. The procedure 
is Summarized in the accompanying table. 


TABLE III 
PROCEDURE FOR A ROTATION EXPERIMENT 


Variable 
Periods ariables 
1 2 
I Group A Group B 
II Group B Group A 


(Note: The table is to be read as follows: During the first 
period, variable 1 would be applied to Group A and variable 
2 to Group B, etc. For three or more variables the table 
could be extended.) 


12. Experimental discovery of causes. The pro- 
cedures outlined in the preceding paragraphs are for 
use in discovering the effects of causal variables which 
are subject to voluntary control. Frequentiy, how- 
ever, we meet a situation in which we have a given 
result but do not know what cause is responsible for it. 
For example, if a rattle develops in an automobile, 
a “causal investigation,” to borrow CcCall’s term, is 
needed to discover its source. Whereas the usual type 
of experiment begins with causes and works toward 
effects, the causal investigation begins with effects 
and works back to causes. There are several ways of 
doing this. One ig to list all possible causes and per- 
form a regular experiment on one after another until 
the one is found which produces the given result, elim- 
inating one cause after another until the true one is 
discovered. A necessary check upon this procedure 
is to remove that one cause from the entire original 
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situation, allowing all the other causes, or variables, 
to operate unchanged, and see whether the given re- 
sult is also removed. This embodies the old principle 
that “Nothing is the cause of a phenomenon in the 
absence of which it nevertheless occurs.” If elimin- 
ating the cause eliminates the results and vice versa, 
the causal relationship between the two is established. 
Another type of procedure for causal investigations 
is to frame a hypothetical cause and apply it to all sit- 
uations possible. If it fits ail situations, it has con- 
siderable claim to truth, but it is not actually proved 
until all the situations or facts are known, and has 
to be kept as a tentative cause or theory. This type 
of procedure is more in the nature of a form of logic 
than a definite experiment, but has proven quite help- 
ful for the advancement of science. For a clear and 
detailed treatment of it, the reader should consult the 
book Introduction to Reflective Thinking, Chapter III* 

13. Correlation. Sometimes it is quite impossible 
to arrange a situation where we can secure the com- 
plete presence or absence of a variable to be studied, 
but instead we have to deal with it in varying quanti- 
ties. For example, we cannot separate children into 
classes of intelligent and not intelligent and note the 
effect of intelligence upon school grades, nor can we 
at will vary the amount of intelligence possessed by a 
given child and note the corresponding change in 
school work. We are forced to take the situation as 
we find it and note whether school work improves as 
we go from students of low to high intelligence. The 
device for determining this relationship is the co- 
efficient of correlation. It really amounts to a statis- 
tical device for summarizing into one numerical term 


*Published by Houghton Mifflin Company, Boston, 1923. 
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the outcomes of as many one-group experiments as 
there are individual subjects included in the computa- 
tion. For the present purpose it is sufficient to say 
that in order to compute correlation it is necessary to 
have numerical data about at least two variables for 
each of several subjects. For example, intelligence 
scores and mathematics grades for each of thirty 
children would enable us to compute the correlation 
between these two variables. This is an outgrowth of 
the older device of the logicians known as the method 
of concomitant variation. It is stated by John Stuart 
Mill as follows: ‘Whatever phenomenon varies in 
any manner whenever another phenomenon varies in 
some particular manner, is either a cause or an effect 
of that phenomenon or is connected with it through 
some fact of causation.”’* 

14. Methods of agreement and of difference. There 
are two types of procedure for the determination of 
truth which, while not strictly experimental, are suffi- 
ciently related to that method to merit brief mention 
here. They are the “method of agreement” and the 
“method of difference.” These methods are quite thor- 
oughly discussed in the book Introduction to Reflective 
Thinking, previously mentioned. The method of agree- 
ment is based on the principle that if two or more 
oecurrances of a given phenomena have only one cir- 
cumstance in common, that one is the cause of the 
phenomena. In other words, the method involves find- 
ing the thing that is common to all cases where the 
given phenomenon occurs, and thereby lIccating the 
cause of that phenomenon. The method of difference 
is based on the principle that “nothing is the cause of 
a phenomenon in whose presence the phenomenon fails 


*Quoted in Introduction to Reflective Thinking, p. 838. 
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to occur.” In other words, it involves finding the thing 
that is always absent when the phenomenon fails to 
take place, and thus determines the cause of the phe- 
nomenon. The use of both of these methods combined 
is naturally preferable to either one alone, since each 
one is a verification or check on the other. This joint 
method consists of finding which variable is always 
present when the phenomenon occurs (agreement) and 
which is always absent when the phenomenon does not 
occur (difference.) An illustration of this procedure 
is involved in Pressey’s investigation of the relation 
between note-taking habits and college grades.* It 
was found that of ten students who had “full notes 
well organized” all ten agreed in that they were “A” 
students: It was found, further, that of twenty stu- 
dents who had “unorganized notes,” seventeen were 
“EK” students and three “A” students. These two 
findings combined present rather strong evidence of 
causal relationship between good notes and good schol- 
arship. This plan of comparing and contrasting ex- 
tremes has possibilities of application in a wide variety 
of situations. 

15. Hidden variables. Every experiment is in a sense 
a game of hide-and-seek, in which the experimenter 
seeks to discover elements in the situation which have 
not been held constant and which might therefore 
vitiate the results. For example, in the case of a 
rotation experiment to determine the relative values 
of direct and indirect methods in foreign languages, if 
the teacher who uses the methods has a conscious or 
unconscious preference for the direct method, this 
fact may affect the outcome of the experiment. Each 


*Pressey, S. L. and others: Research Adventures in Univer- 
sity Teaching. Public School Publishing Company, Blooming- 
ton, Illinois, 1927, page 6. 
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experiment has its own particular hidden variables of 
this sort, but a few common ones that tend to come 
up in many types of studies are as follows: (1) Un- 
equal amounts of time allowed for the two variables to 
operate. (2) Unequal interest of the subjects during 
the time the variables are being applied. (8) Unequal 
vigor or enthusiasm on the part of the teacher or ex- 
perimenter during the application of the two variables. 
(4) Unequal skill or ability to apply the two variables 
on the part of the teacher or experimenter. (5) Un- 
equal opportunity for further progress at the time the 
two variables are applied. (6) Unequal amounts of 
preparation for or familiarity with the two variables 
on the part of the subjects to whom they are applied. 
(7) Unequal amounts of transfer from one variable 
to the other form one experimental period to the next. 
(8) Unequal adaptation of the tests used to the nature 
of the outcome from the two variables. It should be 
the ideal of every experimenter to detect any and all 
such hidden factors as these and to control them, or 
else to measure or estimate their influence upon the 
result and thus make appropriate deductions. Simply 
to point out their existence is a great service, in that 
‘it enables later investigators to discover and guard 
against them. ; 

16. Duplicate experiments.. There are too many 
weaknesses in experimental procedure on educational 
problems at the present time to justify putting com- 
plete faith in the outcome of a single experiment how- 
ever careful the experimenter. There is need for doing 
the same experiment over and over with different sub- 
jects, by different experimenters, using different ma- 
terials, testing in different ways, or carrying the pro- 
cedure through different periods of time. If several 
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such experiments yield uniform and consistent results, 
they justify a much more sweeping and universal gen- 
eralization. If, however, they sometimes yield results 
one way and sometimes another, this disagreement 
helps to define the limits to which the conclusions 
may apply. 

17. Records. If the results of an experiment are 
to be of service to society, sufficient records should be 
kept not only to enable later readers to judge the truth 
of the conclusions but also to enable them to repeat 
the entire procedure of the experiment in exactly the 
same fashion. This necessitates keeping complete and 
detailed records of the study as it progresses. It is 
better to record an occurrence while it is vividly before 
the senses than to record it from memory after a 
period has elapsed. The record should include infor- 
mation about the history of the situation leading up 
to the experiment as well as the dates, persons, groups, 
materials, tests, directions, accidents, interruptions, 
absences, unusual phenomena, etc. etc. In short it 
should contain anything and everything that might 
conceivably have any relation to the investigation, 
since it may develop that the validity of the whole 
experiment will hinge upon a single one of these ap- 
parently insignificant details. 

18. Measurement. It is not our intention here to 
treat measurement in general, since that is done in 
another place, but rather to point out a few matters 
of special importance in experimental studies. There 
are seven of these considerations: (1) Be sure there 
is a “measurement situation” provided. That is, 
be sure that due provision is made to check up on 
what you want to check up. Promiscuous or haphaz- 
ard administration of tests is costly and ineffectual 
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unless a plan of campaign is made to be sure the meas- 
uring instruments apply to what is to be done. (2) 
Measure initial status as well as final status. An ex- 
periment seeks to determine the change produced by 
different variables, and change cannot be determined 
unless the status is measured both before and after 
the application of the variable. A short-cut that is 
sometimes justifiable is to assume that the initial test 
would be zero for each variable or else that the two 
would be equal, and then to treat the final test scores 
as if they were all change. Needless to say this 
short cut should be employed with caution. (38) Re- 
fine the measuring technique by means of technical 
instruments of precision wherever they are available 
and the thing measured is of a sufficiently delicate 
nature. Otherwise, there may be changes present 
which cannot be detected “with the naked eye” but 
which are nevertheless important. (4) Measure re- 
sults by several different kinds of tests. What is 
revealed by one type of test may be different from 
that revealed by another. One test may be of such 
a nature as to favor one of the variables more than 
another, and if only one is used, a misleading impres- 
sion is produced. When using several tests to measure 
results, it is well to compute the results for each one 
separately, as if it were a separate experiment, rather 
than to throw all the tests together into a single com- 
posite. The latter plan throws away a great deal of 
valuable information which can be secured by keep- 
ing the tests separate. (5) Be alert for qualitative 
results as well as quantitative. There are usually 
a number of non-statistical byproducts of an experi- 
ment which it would be folly to ignore. (6) Measure 
not only the amount of change produced but the 
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amount of the variable applied to produce that change. 
(7) Measure results not only at the end of a long- 
term experiment but also at intermediate points during 
its progress. If only the long-term measurement is 
applied, it is not known what the relative positions 
occupied by the two variables at all times during the 
experiment were. It is valuable to know not only 
which won in the end but how the race stood before 
the final sprint. Furthermore, some fruits of an ex- 
periment are better measured directly at the time than 
weeks or months later when they have become diffused 
and confused with the results of numerous other var- 
iables within or outside of the experiment. 

19. Statistical processes. The subject of statistical 
technique is treated elsewhere, and only a few words 
are necessary here regarding the statistics of the ex- 
periment. In any experiment which involves individ- 
ual differences (and that includes practically all educa- 
tion experiments) the use of statistics is necessary in 
treating the results. The essential steps are as fol- 
lows: (1) Compute the mean, or average, change pro- 
duced by each variable and subtract these to obtain 
the average difference. (2) Compute the standard 
deviation of the changes produced by each variable 
and from these determine the standard deviation of 
each mean, in order to know how certain a second 
performance of the experiment on a different sampling 
of subjects would be to yield the same results. (3) 
From the standard deviations of the two means, 
derive the standard deviation of the difference in 
order to know its reliability. (4) Compare the dif- 
ference with the standard deviation of the difference 
as.a test of its significance. In order to have “prac- 
tical certainty” that one variable is really more effec- 
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tive than the other, and that the difference is not due 
to the sampling of subjects used, the difference should 
be 2.78 times its standard deviation. This procedure 
is all outlined very completely in McCall’s How to 
Experiment in Education, and is also treated more ex- 
-tensively in the chapter on statistical technique. 


EXERCISES AND QUESTIONS 


1. Why is the experimental method the backbone of science? 
What is the greatest difficulty in experimental research? 


8. What kinds of problems lend themselves best to short-term 
experiments? What kinds require long-term experi- 
ments? 


4. Tell how you would compare the worth of oral and written 
spelling drill by means of a one-group experiment? A 
parallel-group experiment? A _ rotation experiment? 
Which is best suited to this problem? Why? 


5. Name as many “hidden variables” as you can that might 
creep into the experiment just mentioned, and tell how 
to deal with each. 


6. Select an article or research report and examine it criti- 
cally for weaknesses in the procedure employed. How 
could these weaknesses be corrected? 

7. Formulate plans for a number of experiments in your own 
field of interest and submit them for criticism to your 
teacher or classmates. 


8. List as many as you can of the essential requisites of a 
good experimental procedure. 
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CHAPTER III 
HISTORICAL TECHNIQUE 


1. Importance of special training. There is a rather 
strong tendency for graduate students who have had 
no special training in statistical or experimental tech- 
nique to turn to historical thesis subjects with the 
idea that anybody can do historical research. Such 
an attitude cannot be too strongly condemned. His- 
torical work has a well formulated procedure or tech- 
nique all its own, and is not a field for amateurs to 
enter lightly. The fact that its materials are not quan- 
titative does not mean that its tests for truth are just 
common sense tests or that its standards for success 
are mere matters of whim. An untrained person is 
as likely to fail in historical research as in any other 
type. Anyone who aspires to write history should 
make a definite study of the best books on the tech- 
nique of historical research. 

2. Truth versus interest or propaganda. Although 
all research aims, or should aim, at the truth, the 
whole truth, and nothing but the truth, the temptation 
to depart from this standard is especially strong in 
historical research. The events of the past are sur- 
rounded with a halo of emotional reactions, patriotic 
sentiments, and personal prejudices which are most 
difficuit to overcome. The purpose of history, as it is 
commonly written, is to shed light upon the present 
or provide guidance for the future. This purpose 
causes a strong urge to introduce good moral lessons 
or socially desirable conclusions from the facts rather 
than the conclusions which are actually true to the 
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facts. For example, it is with difficulty that the 
historian breaks away from the story of Washington 
and the cherry tree, because it is to dear to our hearts, 
and has such a splendid (?) moral lesson. Next to 
the factor of moral influence as an enemy of historical 
truth is that of interest. By exaggerating or dis- 
torting facts the story may be given a much more in- 
teresting flavor than that attached to the drab reality. 
A mass of highly interesting material about early 
American schools or teachers may be enjoyable fiction 
but poor history. The writer of fiction has a great 
advantage over the research worker in regard to in- 
terest, and the latter should consistently avoid being 
drawn into a contest in which interest is the measure 
of success. A fact may be uninteresting or even of 
bad moral influence, but, if it is really a fact, the 
historian has no license to omit or to distort it. 
His object should always be to present as complete 
and reliable a picture of past events as is possible, 
regardless of the moral or social implications. 

3. Getting access to sources cf data. Before under- 
taking a historical study, the investigator should con- 
sider thoroughly his facilities for securing the needed 
information. Since the past is already gone, it is 
impossible for the investigator to observe it. Conse- 
quently he must rely upon such remains or documents 
as have survived, or else upon the memories of per- 
sons who observed the events which are to be recorded. 
Getting a few fragments of the past is not sufficient 
for adequate historical work. It is necessary to secure 
enough fragments to establish at least the general 
outline of the whole, and the quality of the work varies 
in proportion to the completeness of the sources con- 
sulted. Every time a gap in history is left to be 
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filled in by the imagination from what occurred before 
and after, the account is materially weakened. The 
problem may necessitate considerable travel on the 
part of the investigator, or considerable expenditure 
of money in order to have rare documents in distant 
places copied or reproduced by photostat process and 
sent to him. The true historian, however, will spare 
no effort or expense to secure an abundance of material 
to use in making his reconstruction of the past, nor 
will he lightly undertake a project before considering 
whether he is able to pay the price required for suc- 
cess in it. 

4, Original versus secondary sources. There are two 
general types of sources from which to write history. 
The first, and the most valuable, is the original, or pri- 
mary, source. It is the actual object, remains, record, 
or document, which has survived from the past. ‘The 
other type is the secondary source, or account of the 
past written by someone who had some kind of access 
to the original source. Secondary sources may, there- 
fore, be classed as hearsay sources, whereas the pri- 
mary sources are those which are consulted directly. 
The ordinary textbooks in history or in the history of 
education are distinctly secondary sources. Some of 
these, however, give footnote citations of the original 
documents from which they were written, and thus 
point the way to primary sources. For an invesigator 
to attempt to write a history of some event by com- 
paring the accounts of that event as given in ordinary 
textbooks, however numerous the sources consulted, 
could hardly be classed as true historical research. 
This does not mean that secondary sources are worth- 
less in historical research. They may provide an excel- 
lent starting point for investigation, and enable the 


52 THE TECHNIQUE OF RESEARCH 


investigator to begin where some previous worker left 
off. Also it may be better to use secondary sources of 
a high grade than primary sources that are very frag- 
mentary and of questionable merit. The chief essen- 
tials in using secondary sources are that they should 
be used with a due consciousness of their limitations, 
and that the reader should be made aware of the nature 
of the foundations upon which the account rests. A 
type of original source which has very high value is the 
legal document, law, or court record. 

5. Lower versus higher criticism. Having located 
documents or historical sources, the next step is to criti- 
cise or evaluate them. This criticism should be of both 
the “lower” and the “higher” type. (1) Lower criti- 
cism refers to the determination of the actual message 
conveyed by the source used. For example, it may be 
a paper that has been much worn, and discolored by 
age so that parts are illegible, necessitating consider- 
able research to restore the missing parts. It may be 
written in a foreign language, secret code, or garbled 
English, whose meaning or interpretation is difficult to 
get. It may have been copied many times, as in the 
case of the Bible, with slight changes with each copy- 
ing, making necessary considerable investigation in 
order to determine what is the true and original ver- 
sion of the document. All these various processes 
which look toward a determination of the true and 
correct text of the document, or to a determination of 
what it actually says, are called lower criticism. (2) 
Higher criticism refers to determination of the author- 
ship, date, age, purpose, dependability, ete., of the 
source, as contrasted with its wording or content. It 
seeks to answer not what was written but who wrote it, 
when, why, where, with how much prejudice, ete. Such 
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a question as whether Homer actually composed the 
Iliad and Odyssey, or whether they were merely 
evolved by wandering bards, is a question involving the 
higher criticism. Until the foundation upon which a 
body of evidence actually rests is known, it is impos- 
sible to know how much faith to put init. Thus higher 
criticism must be accomplished before the results of 
the lower criticism are of historical value. Historical 
research is consequently something of a cycle, involv- 
ing the use of historical technique to evaluate docu- 
ments and then the use of those documents to shed 
light on historical facts. In other words, before we 
can learn history from a document, we must learn the 
history of that document. 

6. Internal versus external criticism. The higher 
criticism of a document may be accomplished both from 
evidence within and without the document itself. (1) 
Internal criticism means judging the document from 
within. For example, the style of one portion of a 
document may be so radically different from that of 
another that there is no room for doubt that two dif- 
ferent persons wrote the document. This type of evi- 
dence has proved that some of the plays ascribed to 
Shakespeare were written in part by other persons. 
Again something in a document may show a premoni- 
tion of future events so strongly as to prove that it was 
written years later than the date given and simply 
dated back. The language and style may shed con- 
siderable light upon the kind of person doing the writ- 
ing, revealing him as ignorant, educated, prejudiced, 
careless, critical, bigoted, dishonest, fairminded, etc. 
The trained historian can learn much about the value 
of a source by such internal evidence. (2) External 
criticism seeks to determine the value of a source from 
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evidences outside. For example, the author of the 
document may be referred to by his contemporaries as 
a scholar, an ignoramus, or a knave. His work may be 
referred to, or his subject treated by another author, 
thus giving an opportunity to compare the two 
accounts, or it may quote or be quoted by other docu- 
ments which shed light upon it. External criticism 
includes also the checking of statements against the 
facts of science and against standards of common sense 
to determine whether the assertions made are possible 
or probable. To summarize, then, a complete criticism 
of a document or source involves both lower and higher 
criticism, from both internal and external evidence; or 
to put it another way, it involves finding what it tells, 
and how much credence its message deserves, checking 
these against evidence within and without the docu- 
ment. 

7. Genuineness versus worth. There are two main 
tests that should be applied before using a given source 
or document. These are genuineness and worth. A 
forged or altered document is clearly of doubtful worth, 
whether the alterations were made deliberately to de- 
ceive, or simply by carelessness of the copyist or 
printer. But genuineness does not guarantee worth. 
A document may be the genuine and original writing 
of a man whose statements are not dependable. How 
to judge the worth of a witness’s statements may be 
answered in part by looking for the following points: 
(1) Who is the witness? If anonymous, his testimony 
is doubtful. (2) Where did he write it? If he wrote 
it at the scene of the event, it is of more value than 
if he wrote it from a distance. (8) When did he write 
it? If at the time, it is preferable to writing made 
from memory at alater time. (4) Why did he write it? 
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If he had some special interest or selfish motive for 
presenting a certain version, his account would be 
questionable. (5) What opportunity did he have to 
observe the events he described? Unless he was in a 
position to get the facts accurately, his account would 
be of small value. (6) Was he capable of using his 
opportunities for gathering the facts? Even though 
he was in a most favorable situation to observe the 
thing accurately and well, if he was deaf, blind, care- 
less, or prejudiced, he would be unable to use the oppor- 
tunities he had. (7) Was he of a sober and accurate 
turn of mind, or was he imaginative and flighty? (8) 
Was he absolutely honest, so that he would report the 
facts according to the best of his ability? (9) Was he 
an eye witness or did he rely on hearsay for his infor- 
mation? In short, in judging a witness to a past event, 
it is necessary to apply all the tests that would be 
applied to a scientist at the present day, and if he is 
not a good scientist, his testimony is to be treated 
accordingly. 

8. Note-taking. Writers on historical research prac- 
ticaily all agree upon the impcrtance of having a defin- 
ite plan of note-taking for the work. Since it is impos- 
sible to generalize or to establish the facts about a 
given event until the evidence is all collected, a note- 
taking system should be such as will preserve the 
essentials of each source until they have all been 
assembled. It should also permit the ready assembling 
of all statements bearing on a given event in one place, 
where they can be before the eye at once. Some histor- 
ians report the practice of writing all statements bear- 
ing upon a given event on the same large sheet to keep 
them all together. Others accomplish the same pur- 
pose by means of cards. It is very important that the 
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notes represent accurately and truthfully the sources 
from which they are taken. Ordinary quotations are 
preferable, as they involve less likelihood of distortion. 
Also quotations are more serviceable as means of giving 
credit for statements made. Care is necessary, how- 
ever, to avoid selecting quotations in which the mean- 
ing, apart from the context, is different from that 
which they have in their original settings. Also, ex- 
treme care is necessary to avoid changing the meaning 
of a statement by slight errors or omissions during 
the copying. A few checkings of copied material will 
reveal the fact that the ordinary person cannot trust 
himself to copy anything accurately, and that all notes 
must be checked back with the original copy from 
which they come. 

9. Establishing facts. Before an event can be estab- 
lished as a fact, there must be evidence for it. Prob- 
ability and reasonableness do not justify a conclusion 
without evidence. A fact is said to be established when 
two or more competent, independent witnesses agree 
upon it. Stress has already been laid on competence, 
but independence deserves special mention. Two wit- 
nesses may report the same story, yet not be inde- 
pendent. One may have heard it from the other or 
both from a third person. Also they might both have 
been deceived by the same circumstances surrounding 
the event. The courts of law have been obliged to set 
up very definite limitations regarding the kind of evi- 
dence which is acceptable, and many of these limita- 
tions are equally applicable to the general field of his- 
torical research. When absolutely conclusive evidence 
is lacking, the historian will simply state his conclu- 
sions as possibilities, taking pains to tell upon what 
kind of evidence they are based. The better the train- 
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ing he has, the fewer dogmatic and unsupported state- 
ments he will make. 

10. Composing the narrative. After the facts have 
been established, the next step is to write them up in 
a form that will make them available, and serviceable 
to others. This phase of the work is more an art than 
a science, but the value of the preceding science is very 
much dependent upon it. Unless the facts are pre- 
sented intelligibly, and more or less interestingly, they 
will be of little worth. As a general rule it is necessary 
to decide whether the narrative is to be written for the 
popular reader or for other historians, since the former 
is interested mainly in the facts and the latter mainly 
in the evidence. The account may be organized in any 
one of several different ways, as chronological, topical, 
biographical, etc., the particular plan depending upon 
the writer and the subject. But whichever scheme of 
organization is employed, the best genius of the writer 
should be devoted to making that scheme compelling 
and effective. Also the narrative may include the 
author’s own philosophy or interpretation of the events, 
provided a clear distinction is made between what is 
fact and what is the author’s interpretation of the fact. 

11. Documentation. After a number of documents 
or other sources have once been critically examined and 
their worth appraised by one careful historian it should 
be unnecessary for this work to be repeated by others, 
except for the purpose of checking. This necessitates 
leaving a careful and complete record of all sources 
consulted and all evidence which has been used in the 
establishing of the facts. To arrive at the conclusions 
laboriously and critically and then put into the records 
only the results of that labor is most inconsiderate, both 
of the reader and later historians. The documentation 
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of a work of history calls for liberal footnotes citing 
sources of evidence and their peculiar merits, as well 
as free references to them in the body of the narrative. 
The merit of a piece of historical writing is to be 
judged in terms of its freedom from unsupported 
statements and of the ease with which the reader can 
check upon statements to see whether the historian 
has properly evaluated the evidence. 

12. Investigating recent cr present educational 
trends. A favorite type of thesis topic for graduate 
students in education is that illustrated by “Recent 
trends in the teaching of algebra,” or “Present tend- 
encies in the teaching of shorthand.” This is a very 
worth while type of problem, but it is often undertaken 
by very crude and unsatisfactory procedures. A com- 
mon plan is to collect from various writers of books and 
articles a number of statements which give those 
authors’ opinions of the trends and to say those are 
the trends. Another common error is to find out some- 
thing about present conditions or practices and not 
even trouble to find out what were the previous prac- 
tices that corresponded to these. The following prin- 
ciples and procedures may help to improve practice in 
this phase of historical research: (1) It takes two 
points to establish a line. Consequently, a trend is to 
be measured by comparing conditions or practices at 
two different periods of time. (2) The further the 
points are apart the less the danger of making an error 
in the general direction of the line through imperfect 
techniques. Thus a study of trends which compared 
two periods separated by fifty years would be more 
reliable than one which covered a period of only five 
years, other things being equal. (8) It is better to 
contrast two periods that are separated in time than 
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two that are adjacent in time. For example, it would 
be better to compare practice in 1890-99 with that in 
1920-28 than to compare practice in 1890-1909 with 
that of 1910-1928. The former is a more sharp con- 
trast than the latter, because it measured extremes 
within the period in question. (4) Frequently a study 
of trends requires a combination of historical tech- 
nique and survey technique, the former to determine 
what the practice was during the earlier period, the 
latter the present period. (5) The conditions and prac- 
tices of each period should be tabulated, or classified, 
according to the same scheme, else they are not com- 
parable. For example, “number of pupils in high 
school” in 1890 and in 1920 must take into considera- 
tion the junior high school movement. (6) The data 
for each period should be stated in statistical or numeri- 
cal form if at all possible, as numbers are likely to 
allow less room for subjective errors than adjectives 
and verbal descriptions. (7) For some problems the 
use of index numbers such as have been devised by the 
business and economics specialists may be used to 
advantage. (8) Textbooks of the two contrasted 
periods may be analyzed and compared as to percentage 
of space given to different topics as a means of com- 
paring educational practices. This would show the de- 
crease of certain kinds of material and the increase of 
others. (9) Professional books, that is, books for the 
teacher instead of the pupil, may be analyzed as to 
space distribution, or as to the authors’ views upon a 
selected set of issues, and results for the two periods 
compared to show trends in educational thinking about 
the particular educational problem. (10) Periodical 
materials written for teachers may be compared to 
show shifts of emphasis in views about the educational 
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problems from one period to the next, and are some- 
times preferable to books because they permit a more 
extensive sampling of authors. (11) The dates of pub- 
lication of books may be used to show trends by show- 
ing the increase or decrease of publications on a given 
subject as compared with some other field. For ex- 
ample, it is possible to note whether the number of 
social science texts published has increased faster than 
the number of Latin or Mathematics texts. (12) The 
number of articles published in periodicals on a given 
subject for two periods may be compared to show its 
increase or decrease in popularity as a subject of dis- 
cussion in the journals. (13) The number of entries in 
the Readers’ Guide, International Index, or United 
States Catalog on a given subject may be counted to 
determine trends as shown by number of publications. 
For example, if the subject of “art’’ showed a larger 
percentage gain in entries in 1925 under “Art Appre- 
ciation” than in 1910, as compared with entries under 
“Painting” or “Design,” we should say there was a 
trend to magnify the importance of appreciation. (14) 
Old examination question whose answers have been 
saved may be given to children of the present, and the 
scores compared to see what change in outcomes of 
instruction has taken place in the meanwhile. (15) 
Questionnaire studies that were made years ago may 
be repeated in identical fashion today and the answers 
compared. (16) Great care should be taken to see that 
the materials chosen to represent the two periods to 
be compared are truly representative of their respec- 
tive periods. 17) The existence of a trend should not 
be taken as a justification of it. It is entirely possible 
that the trend may be in a given direction when the 
actual scientific facts would justify the opposite course. 
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13. Historical errors. In closing this brief discussion 


of historical technique it seems fitting to give a list of 
typical errors which are commonly made by historical 
workers and which are especially to be avoided*: They 
are as follows: 


(1) Statement of facts without evidence. 
(2) Misstatement of facts. 

(3) Wrong interpretation of facts. 

(4) False conclusions from the data. 

(5) Inaccurate quoting. 

(6) Substituting probability for fact. 

(7) Argument from silence. 

(8) Argument from anology. 

(9) Plagiarism. 


*Mentioned in lectures given by Dr. M. W. Jernegan, Univer- 


sity of Chicago, Department of History. 


10. 


Ai. 


HXERCISES AND QUESTIONS 


How does historical research resemble the work of the 
judge and jury in a trial? 


What is to be gained and what lost by writing true history 
and no other kind? 


How does historical research compare with other types of 
research as to feasibility in your own particular case? 
For students of education in general? 


How does the practical value of historical research in edu- 
cation compare with that of experimental research? 


What are the principal materials which you could use if 
you were to undertake a historical problem? Are these 
adequate? : 

How would you judge whether a source was original or 
secondary? 

Is it possible to write a complete history of any past event? 
Explain. 

Why are footnotes more needed in historical research re- 
ports than in experimental ones? 

How do studies of present or recent trends compare in 
value with other kinds of historical research. 

Collect a few references to historical investigations and 
read at least one such report, trying to identify each of 
the steps employed by the person who made the study. 


Formulate what you consider the essentials of good his- 
torical research. 
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READINGS 


Columbia Associates in Philosophy. Introduction to Reflective 
Thinking. Houghton Mifflin Company, Boston, 1923. 

Chapter VIII, ‘“How Reflective Thought Deals with the 
Past, as Illustrated through the Criticism of the 
Pentateuch.” 

Fling, Fred M. The Writing of History. Yale University 
Press, New Haven, Connecticut, 1920. 
An excellent statement of the historical technique. 
Johnson, Henry. Teaching of History. The Macmillan Com- 
pany, New York, 1925. 

Chapter I, “What History Is”; Chapter XIV, “School 
History and the Historical Method.” 

Keatinge, M. W. Studies in the Teaching of History. Adam 
and Charles Black, London, 1910. 

Chapter II, “Scientific Method in History and the Prob- 
lems of the School”; Chapter III, “Contemporary Docu- 
ments as a Basis of Method.” 

Langlois, C. V., and Seignobos, C. Introduction to the Study 
of History. Henry Holt and Company, New York, 1898. 

Translated from the French; a good treatment of his- 

torical method. 
Ogg, F. A. Research in the Humanistic and Social Sciences. 
The Century Co., New York, 1928. 
Schouler, James. Historical Briefs. Dodd, Mead and Company, 
1896. 
Contains material on the methods employed by historians. 
Vincent, J. M. Histovical Research. Henry Holt & Company, 
New York, 1911. 
The book is “An outline of theory and practice.” 


COA PAB REV. 
PSYCHOLOGICAL TECHNIQUE 


1. Psychology as a science. Psychology in times past 
was closely associated with philosophy. Its method was 
largely that of rational criticism and analysis rather 
than observation and experiment. Speculation was 
used to build up elaborate structures of theory which 
would not bear the test of objective verification. The 
separation of psychology from philosophy is now recog- 
nized as imperative. It is time for the building of 
“armchair” theories by philosophical methods to give 
way to strictly scientific ways of studying psychologi- 
cal phenomena. The common notion that one system or 
school of psychology is as good as another is a result 
of the fact that for a long time each school of psychol- 
ogists started with its own major premises and pro- 
ceeded to speculate, with internal consistency as the 
principal test of the soundness of the theories, each 
naturally getting different results from the others. 
With the perfection of more objective and scientific 
procedures the day is rapidly approaching when we 
shall have one science of psychology instead of several 
systems of philosophy about things psychological. Some 
of the principal ways of attacking this field scien- 
tifically are given in the following paragraphs. 

2. The experiment. The psychologists seem quite 
agreed that the experimental method is a central one 
in their science. It has been stated that “experiments 
are far more reliable than experience” as guides to the 
understanding of behavior. They involve subjecting 
the individual to a well-controlled set of conditions, 
applying a desired stimulus, and noting the resulting 
behavior. Such a procedure is infinitely better than 
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trusting the uncontrolled observations of one or more 
individuals as a basis for conclusion. Since the pro- 
cedure for an experiment is largely the same in prin- 
ciple in one science as in another, the chapter on experi- 
mental technique should be reviewed in relation to the 
present one. 

3. Observation. Some of the early writings on psy- 
chology were based on what has been called “the 
method of anecdote’, or the collection of a number of 
stories, incidents, cases, or experiences of a psychologi- 
cal nature which had come to the various persons re- 
porting them. Thus a collection of anecdotes about 
behavior of lower animals which indicated unusual rea- 
soning ability might be used as a foundation for a 
conclusion regarding the intelligence of lower animals. 
Such methods have proven unsound and misleading. 
The anecdotes are usually based on unsystematic ob- 
servation, often by an untrained observer, and without 
adequate records being made at the time to guarantee 
accurate memory of the facts. To take its place there 
has been developed an elaborate technique for careful, 
continuous, and prolonged observation by highly 
trained observers. For example, Thorndike studied 
animals in this direct way in some of his early work, 
and Watson took opportunity to observe infants in 
maternity wards and hospitals in a way that would 
insure much more accurate data than that which 
mothers could be relied upon to report about their own 
children. Rather than to collect anecdotes from many 
observers, it is better to let one observer make many 
systematic observations of the phenomena in question. 
This will frequently involve the use of special labor- 
atories, and even the invention of new instruments or 
apparatus to aid in making the observations. A little 
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study of the catalogs of companies which manufacture 
psychological equipment will sometimes go far to sug- 
gest available instruments which may be used, or to 
give ideas for the making of new ones. Further sug- 
gestions on the technique of observation are given in 
a separate chapter. 


4, The conditioned reflex. Many forms of behavior 
are difficult to observe because they are internal rather 
than external. It frequently happens, also, that lan- 
guage is not available as a means of getting a report 
on those forms of behavior, as in the case of studies on 
animals, infants, feebleminded people, or foreigners. 
The conditioned reflex is a useful device in such situa- 
tions. For example, if a bell is sounded and a bright 
light flashed into the eye simultaneously for a num- 
ber of times and. then the bell is sounded alone with- 
out the light, it is found that the pupil of the eye 
contracts just as if the light had been flashed upon 
it. Similarly, other combinations or associations can 
be effected. This makes it possible to graft upon an 
internal, and therefore hidden, form of behavior, some 
very objective and concrete form of external behavior 
which may become a signal or an indication that the 
inner response is taking place. This device substi- 
tutes the observation of an explicit or outward re- 
sponse for an implicit or non-muscular one, and makes 
unnecessary the resort to introspection or verbal re- 
port upon the internal reactions. The method is par- 
ticularly good as a method of studying sensory dis- 
crimination, but it has also many other applications. 
Further treatment of its use may be found in the 
writings of Watson* and others. 


*Particularly Watson, J. B. Psychology from the Stand- 
point of a Behaviorist, pp. 28-38. 
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5. Neurology and physiology. Some of the most 
illuminating studies in the field of psychology have 
been based upon actual dissection of the brains, nerv- 
ous systems, sense organs, etc., of men or animals. 
One important part of the task of understanding be- 
havior is to study the mechanism by which that be- 
havior takes place, of which, naturally, the nervous 
system is an important part. When this method is 
combined with that of experimentation, it proves quite 
fruitful for some purposes. For example, local stimula- 
tion may be applied to a particular nerve ending and 
the consequent muscular response noted, or, what is a 
bit more severe, surgical operations may be made upon 
animals, removing certain portions of the brain or 
severing nerve connections and observing the conse- 
quent changes in behavior. Of course, it is impossible 
to study all phases of psychology by the neurological or 
psysiological methods, but it is also true that it would 
be impossible to dispense with them. | 

6. Pathology. The study of abnormal or pathological 
cases often sheds a great deal of light upon the ordi- 
nary or normal cases. An insane person may manifest 
in an exaggerated degree a trait or form of behavior 
which is present in less degree in ordinary persons but 
which has been overlooked; or a person suffering from 
paralysis may, by his loss of a certain power or ability, 
call attention to things whose importance has not been 
fully appreciated before. Just as a man learns more 
about automobiles when they give trouble than when 
they run smoothly, so we discover many things about 
human nature by the study of the decay or disintegra- 
tion of mental powers that we can never learn by the 
study of those powers when they are in perfect condi- 
tion. Abnormal psychology is, therefore, more than a 
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field. It is a method. It is a field in that it represents 
many urgent problems that are in need of solution for 
the sake of the persons involved; but it is a method in 
that we may use these cases as a means of getting at 
the more general laws of behavior for normal persons 
as well. Some of the important educational advances 
of recent years are due to lessons learned in dealing 
with delinquents, feebleminded children, or school fail- 
ures, lessons which were equally applicable to all chil- 
dren. A well-known educator has said that some of 
our best schools are to be found in the institutions 
which are designed for abnormal or problem children. 
The psychological clinic, called upon as it is to solve the 
difficulties of numbers of maladjusted individuals, can 
give us some of our best information on human nature 
in general. Similarly, our “study clinics’, wherever 
they exist, may add much to our knowledge of the 
nature of the learning process and the psychology of 
particular school subjects and tasks. It is not our in- 
tention here to encourage amateurs to undertake in- 
vestigations involving diagnosis and treatment of ab- 
normal individuals, but rather to encourage more effort 
on the part of experts already so engaged to generalize 
their findings and to put their information to use for 
the benefit of mankind as a whole. 

7. The study of animals. There are several factors 
which make it desirable to study animal behavior as a 
means of getting psychological insight. (1) There are 
many problems involved in training and dealing with 
animals which deserve investigation on their own mer- 
its. (2) Since man represents an advanced stage in the 
evolutionary process, it is profitable to understand that 
from which man’s behavior has sprung. (3) Animal 
behavior, because it is simpler than that of man, is for 
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that reason easier to study than man’s. (4) The ab- 
sence of higher thought processes and of self-consci- 
ousness on the part of animals makes it easier to get 
examples of the lower types of responses from them 
in a state of purity not always possible in man. (5) 
It is easier to control the life and environment of ani- 
mals than of men, since they may be imprisoned, 
starved, mated, etc., at will. (6) More drastic meas- 
ures may be employed with animals than are per- 
mitted with man, even extreme cruelty, permanent 
disability, surgical mutilation, vivisection, and death. 
(7) Their shorter life span enables an investigator to 
study infant, mature, and senile behavior all in the 
span of a few days or months whereas human beings 
would require years. One important caution is in order 
here, that the drawing of inferences from animal to 
human behavior must be done with the utmost care. 
What is true of rats or cats may not be true of men or 
women or boys or girls. The processes by which dogs 
learn a maze may not be the same as that by which 
children learn to write. It may be well to perform an 
experiment first on animals to learn as much as pos- 
sible from that source, but it will be necessary to repeat 
it with human subjects as a final check and verification, 
if conclusions of significance are desired. 

8. Genetic studies. Several of the advantages of the 
comparative method, or study of animals, are also pres- 
ent in the genetic method, or the study of children and 
of primitive man. Children and savages are more naive 
or unsophisticated, and hence, respond in a simpler way 
than civilized adults. Since much of psychology is a 
matter of inborn tendencies, what is more natural than 
to go back to that period in man’s existence when these 
tendencies were becoming imbedded in human nature, 
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or when they functioned with a lesser admixture of 
the learned forms of behavior which characterize civil- 
ized life? Or what is more natural than to study hu- 
man nature as it is manifested in small children, rela- 
tively free from the artificialities produced by environ- 
ment and training? Since it is so difficult to secure for 
study primitive peoples of the present day who are 
really typical of those who furnish the genetic back- 
ground of present civilized peoples, the greater amount 
of genetic studies now are being devoted to children. 
There are several points which should be considered by 
those who make such investigations: (1) Studies 
should begin at, or in some cases before, birth, in order 
to be able to distinguish more closely between learned 
and acquired behavior. (2) The dates and circum- 
stances of the appearance of particular responses for 
the first time should be fully recorded. (8) The ob- 
servations should be continuous rather than intermit- 
tent, if possible. (4) A few intensive studies ordinar- 
ily prove more valuable than several superficial ones. 
(5) It is better to observe the same children for several 
years than to make observations of several different 
age groups at one time, and let these represent the 
course of development which all children are supposed 
to follow. (6) Ordinarily the studies are made best 
by persons who are not related by blood to the children 
who are studied. Mothers are seldom able to observe 
and interpret scientifically the behavior of their own 
children because of their parental feelings and bias. 
(7) On account of the complexity of behavior after the 
period of early infancy, it is ordinarily necessary to 
select for study some single phase of the child’s con- 
duct and not attempt to record everything he does. 
(8) It is often necessary to reverse the order of genetic 
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studies, going back into the history of an individual 
child to find the roots of a trouble or behavior disorder 
from which he is suffering. The mental hygiene ex- 
pert is often much concerned with the genesis of the 
trouble from which his patient suffers. 

9. Psycho-analysis. In recent years a number of pro- 
cedures which are based upon the general concept of 
the unconscious have been evolved for studying human 
nature. The idea is that we have, in a sense, two 
selves, the conscious and the unconscious, and that 
there are many powerful impulses or motive forces 
working in us which we will not admit or recognize as 
being present. For example, we may suppress our sex 
desires so that we do not know that we have them, 
but we go on giving them expression in a disguised 
way through interest in art, religion, or charity. Or 
our suppressed desires may gain expression through 
dreams, day dreams, slips of the tongue, automatisms, 
etc. A rather wide variety of research procedures have 
been evolved which rest upon this line of reasoning. 
For example, dreams, day-dreams, etc., have been rec- 
orded and studied as a means of discovering the roots 
of mental disorders suffered by patients in institutions 
for the treatment of mental diseases. Likewise, chil- 
dren’s daydreams, slips, and careless errors have been 
studied for light upon personality disorders from which 
they suffered. Similarly, association tests have been 
devised to detect lying, and even to secure from a per- 
son information which he did not know he had in his 
possession. The method of psycho-analysis involves 
simply the observation of behavior which we indulge 
in when we are “off our guard” as a means of learning 
what are the dominant impulses that we have failed to 
recognize as present in us. 
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10. Introspection. <A lively, and sometimes bitter, 
debate is going on among psychologists as to the recog- 
nition that shall be given to introspection as a method 
of studying psychology. The chief objections to intro- 
spection are as follows: (1) That introspection inter- 
feres with the state of mind that is being introspected. 
(2) That introspection assumes the existence of con- 
sciousness, whereas one school of psychologists main- 
tain that only behavior is subject to scientific study. 
(83) That introspection is individual, in that no single 
person can introspect any mind besides his own, and 
hence that adequate standards for comparing mental 
states are lacking. (4) That introspection is impos- 
sible as a means of studying infants, animals, or per- 
sons who lack language. (5) That introspection breaks 
down where it is most needed, in the study of complex 
mental states; the fact that it is so subjective makes 
it questionable when the processes studied are of an 
intangible sort. (6) That what a person thinks to be 
his mental state or his dominant motive is an unreliable 
index to what it really is; he is quite often self-deceived 
and unable to make accurate analyses of his own state 
of mind. 

Arguments advanced in favor of introspection are: 
(1) That to study behavior only and to omit a study of 
consciousness is to exclude from consideration an im- 
portant portion of the field of psychology. (2) That 
introspection is simply self-observation, and in many 
cases may be as objective as the observation of others. 
(3) That whereas observation of behavior is only a 
means of inferring the nature of the accompanying con- 
scious states, introspection is a direct observation of 
those conscious states themselves. (4) That intro- 
spection is merely a “reflection upon” experience, or 
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becoming interested in its relation to other experi- 
ences, and is not materially different from the forma- 
tion of judgments about any other kinds of facts. 
(5) That it need not be merely an individual method, 
since several persons may observe themselves and re- 
port their observations verbally. (6) That the intro- 
spection does not banish the mental state introspected, 
but rather that it simply observes that mental state as 
it gradually fades out and gives way to the introspected 
mental state; in other words, that the two processes or 
mental states linger together in the mind for a time, 
during which one may be studied by the other. (7) 
That verbal reports upon the general nature of the 
behavior may tell more than detailed or minute objec- 
tive analysis of the behavior itself would tell. (8) 
That introspection may be useful as a source of sugges- 
tions or hypotheses that may be studied by more refined 
methods. (9) That introspection may yield valuable 
rough or approximate observations and pave the way 
for other studies involving more precise methods. (10) 
That a due appreciation of its limitations would elimin- 
ate many of its disadvantages. 

11. Testing. Many abilities, mental states, or forms 
of behavior that cannot be observed directly can be 
studied very objectively by means of tests. Given suf- 
ficient resourcefulness and ingenuity, a test can be 
constructed to measure almost any psychological phe- 
nomenon imaginable—attitudes, instincts, sensory dis- 
crimination, self-consciousness, suggestibility, memory, 
reasoning, and many others. These need not be, 
though often are, of a verbal or linguistic nature. The 
service rendered to the science of psychology by the 
Army Alpha and Army Beta tests is sufficient justifica- 
tion for the favorable reception which psychologists 
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give to tests as psychological instruments. Some fur- 
ther discussion of the technique of testing may be 
found in the chapter on measurement technique. 

12. Learning curves. Considerable research has been 
done along the line of keeping detailed records of prog- 
ress in learning various tasks, as ball-tossing, mirror- 
drawing, or mastering foreign vocabulary. Yet much 
remains to be done in this line. Learning curve studies 
may well be made for other tasks that have not been 
previously investigated, or for the same tasks under 
different circumstances as regards motivation, length 
of practice periods, etc. Until we know in detail all 
there is to know about the nature of learning for each 
school subject, exercise, or lesson, there will still be 
room for careful scientific work of this type. Care 
should be taken, however, to avoid duplicating studies 
that have been previously published, and to be sure 
that the particular example of learning to be investi- 
gated has sufficient social utility to warrant the ex- 
penditure of effort. 

13. Individual differences. Much of the effort of 
psychologists has been spent in learning the character- 
istics of human beings in general, but, on the other 
hand, and particularly in recent years, considerable 
effort has been spent in learning how individuals differ 
one from another. Also, special groups have been 
compared to determine group differences, as between 
boys and girls, white and colored, city and country 
residents, etc. In reality the study of individuality and 
the study of the universal traits of human nature are 
very closely similar. Statistical method is fundamental 
to each. By means of it we can bring many individual 
cases together and discover the central tendency and 
degree of dispersion, or we may subclassify the cases 
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and note the uniformities running throughout all cases 
within a group, and how they differ from other groups. 
The fact that human behavior is so variable makes one 
of the fundamental problems of psychology the search 
for norms and the discovery of the amount of depart- 
ure of individual cases from these norms. 

14, Applied psychology. Much psychological re- 
search is needed to coordinate and correlate practice 
with the theory which we already know. This work 
may proceed from theory to practice, or from practice 
to theory. For example, the fundamental generaliza- 
tions arrived at by the pure psychologists may be 
tested out in practical situations to determine their 
specific applications. It is not enough to know the 
general laws of learning; we must know also some- 
thing about the laws of learning arithmetic, spelling, 
or music. The study of such a problem requires a 
discovery of the nature of the bonds involved in mas- 
tery of these respective subjects, and the effectiveness 
of the various instincts or innate tendencies as motives 
for the learning. It would be most unfortunate if the 
results of experiments on ball-tossing or on memorizing 
nonsense syllables were “transferred” bodily to the 
learning of handwriting or history. The other angle of 
approach to applied psychology is that from practice to 
theory. For example, specific problems in classroom 
procedure may be attacked by scientific procedures, and 
when several such problems have been solved the re- 
sults may justify the inductive formulation of a gen- 
eral psychological principle or law. Likewise, observa- 
tion of classes at work analyzing textbooks or reading 
books on educational practice may raise problems of a 
psychological nature which become food for serious 
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thought by those who are not primarily educational 
practitioners. 


10. 


2h. 


EXERCISES AND QUESTIONS 


eos does psychology resemble philosophy? How does it 
iffer? 


In what way does psychology differ from the physical and 
biological sciences as regards is. techniques? 


Describe a situation in which the conditioned reflex could 
be used for research purposes. 


What similarities do you see between the neurological and 
the pathological approaches to psychology? What dif- 
ferences? 


Mention some problems of educational puaeAnce which 
might be attached by the study of animals. 


What relation does the genetic viewpoint have to psycho- 
analysis? 


After reading a number of authors’ viewpoints on intro- 
spection, formulate what you consider a tenable position 
regarding it. 


Examine a number of psychological investigations and 
aeeHy them as to the types of psychological technique 
used. 


What is your reaction to the criticism that psychologists 
are putting too much faith in testing? 


Ouline the major types of educational problems that seem 
to call for psychological techniques. 


How can psychological research be made to contribute 
more to education? 
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Chapter I, “Aims of Applied Psychology”; ChapterII, 
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Freeman, F. N. Mental Tests. Houghton Mifflin Company, 
Boston, 1926. 


Whole book. 
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nam’s Sons, New York, 1922. 
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technique. 
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ton & Company, New York, 1916. 
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Jastrow, Joseph. Fact and Fable wn Psychology. Houghton 
Mifflin Company, Boston, 1900. 
Chapter II, “The Problems of Psychical Research.” 
Judd, Charles Hubbard. Psychology of High School Subjects. 
Ginn & Company, Boston, 1915. 
Pp. 10-16, “Methods of Collecting Psychological Material.” 
Klemm, Otto. A History of Psychology. Charles Scribner’s 
Sons, New York, 1914. 
Traces the development of psychology, its schools and 
methodology. 
Moore, Jared Sparks. The Foundations of Psychology. Prince- 
ton University Press, Princeton, N. J., 1921. 
A critical analysis of the various schools of psychologists 
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ton Mifflin Company, Boston, 1916. 
Presents in detail the technique for the Stanford-Binet 
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Houghton Mifflin Company, Boston, 1918. 


Chapter II, “Methods of Scientific Study of Children.” 
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Washburn, M. F. The Animal Mind. The Macmillan Com- 
pany, New York, 1917. 


The book is rich in material on psychological technique, 
paricularly as regards comparative psychology. 


Watson, John B. Psychology from the Standpoint of a Be- 
haviorist. J. B. Lippincott Company, Philadelphia, 1919. 


Chapter I, “Problems and Scope of Psychology”; Chap- 
ter II, “Psychological Methods.” 


Whipple, Guy M. Manual of Mental and Physical Tests. War- 
wick and York, Baltimore, 1910. 


Williams, James K. The Foundations of Social Science. Al- 
fred A. Knopf, New York, 1920. 


Chapter XXIV, “The Methods of Social Psychology.” 


Most textbooks on psychology have one or more chapters on the 
scope and methods of psychology. 


CEUAT CERRY. 
CASE-STUDY TECHNIQUE 


1, Nature and purposes of the case study. As the 
term implies, the case study has to do with individual 
persons or situations rather than with large groups or 
collections of persons or situations. It is a study of 
individual cases rather than cases in the aggregate. 
It is more concerned with the individual than with 
society as a whole. It is interested in discovering 
what is unique rather than what is characteristic of 
large numbers of persons or situations. It is more 
concerned with the exceptional than with the typical. 
It attempts to discover everything that is significant 
in the life or behavior of an individual person rather 
than something that is significant for all persons. A 
few of the special educational situations which call 
for case studies may be mentioned here: 

(1) The failing child. His failure cannot be under- 
stood or remedied except in the light of a careful 
study of the factors which are responsible for his 
failure, and these may be different from those which 
cause failure for other individuals. The director of 
& study clinic must necessarily be somewhat of a 
case worker. 

(2) The disciplinary case. Problems of morality and 
ethical character have been a most frequent stimulus 
to the making of case studies. Those who deal with 
juvenile, as well as adult, offenders have found it 
absolutely necessary to secure full and complete in- 
formation about those whose behavior is causing the 
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problem. Specialists attached to courts and institu- 
tions for correction, as well as school officials who 
deal with numbers of disciplinary cases, have been 
obliged to evolve special policies and procedures for 
giving each individual the special consideration de- 
manded by his case. 

(3) Maladjusted personalities. The mental hygiene 
specialist is constantly dealing with persons whose 
emotional behavior is non-typical. One child may be 
abnormally shy and timid, another unduly aggressive, 
a third inclined to excessive fears, and a fourth to 
fits of melancholy which interfere with his happiness 
and usefulness. These cases all call for sympathetic 
and scientific study, looking toward corrective treat- 
ment, and each child’s case must usually be dealt with 
separately, because few cases are exactly alike. 

(4) Students in need of educational and vocational guidance. 
Fitting an education to a child, or adjusting a child 
to his proper sphere in the vocational world, calls for 
a thorough and all-round study of that particular 
child. General principles or universal formulae cannot 
be applied automatically or in a blanket way to all 
children alike. Before deciding whether to reeoommend 
law, medicine, plumbing, or piano-tuning to a child 
as an occupation, it is necessary to know what are 
the traits, characteristics, capacities, and defects of 
that child which would make for success or failure in 
these respective callings. 

To summarize, the case study arises from practical 
needs rather than from the field of pure science. It 
is the device by which the practitioner applies the 
science that is already known to the cases or indi- 
viduals with which he works. Case studies in medi- 
cine are more commonly made by the practicising 


CASE-STUDY TECHNIQUE 81 


physicians than by the scientific investigators, and 
social case studies are more often made by the actual 
social workers than by those students and investi- 
gators who are primarily interested in advancing the 
science of sociology. Similarly, case studies in 
educational work are naturally of more concern to 
teachers, administrators, school counsellors, mental 
hygiene specialists, etc., than to scientists working 
outside of the actual school situation. 

2. Selection of cases to be studied. As a general rule, 
the selection of cases to be studied is not left to the 
case worker’s choice, because the actual field situation 
forces him to study particular cases whether he will 
or not. A physician does not choose whom he will 
treat, but rather his attention is demanded by those 
who happen to be sick. And yet there are times when 
not all cases which really need attention can be cared 
for because of a shortage of workers or other facili- 
ties which are required—a condition quite prevalent 
in education work. In such a situation, it is necessary 
to begin where the need is greatest, and where the 
greatest good can be done. The following considera- 
tions may be of some help in determining which cases 
to select for study: (1) Select on the basis of the 
social worth of the proposed study and treatment. 
(2) Select on the basis of the amount of advancement 
to science which will come from the study. (8) Select 
incipient rather than advanced cases, so that malad- 
justments may be corrected before they go too far 
to be cured, and before they do much harm. (4) 
Sometimes it is best to select the cases which vary 
most widely from the normal or typical, in order to 
have available for study a rather large or drastic 
sampling of the peculiarity or unusual phenomenon. 
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(5) Select a few cases for very intensive study rather 
than making a superficial study of a great many cases. 
It may be better to cure a few people out of a host 
of wounded on a battlefield than to give temporary 
and superficial relief from pain to a large number. 
The value of a case study depends upon intensive 
more than extensive analysis and observation. 
3..Diagnosis. Each case study should be a solution 
of a definite problem. It should not attempt a com- 
plete survey of the individual in general but rather 
a determination of the cause of the specific trouble 
from which he suffers. Diagnosis is, therefore, a very 
fundamental purpose in a case study, for nothing con- 
structive can be done until the cause of the trouble 
is known. Case work is exactly like the work of the 
physician in this respect. Diagnosis must come before 
treatment. A search for symptoms, or outward man- 
ifestations of the maladjustments, comes first. This 
includes a careful inventory of the specific peculiarities 
which are present. Then it is in order to work back 
from these to the more fundamental roots of the dis- 
order, or to locate the causes which are not so clearly 
revealed. It is not always easy to distinguish between 
the fundamental disturbance and some of its results, 
and this fact should cause an attitude of caution 
against attempting to remove the symptom without 
correcting the disease. The art and technique of 
diagnosis will vary according to the particular case, 
and some cases will require the services of highly 
trained medical or psychological experts while others 
may be within the scope of possibility for the class- 
room teacher. Sometimes diagnosis is greatly aided 
by the use of a list of common causes of maladjust- 
ment, with the behavior characteristic of each, against 
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which list the present case may be checked. More 
often, however, the task of diagnosis constitutes a 
major problem in reasoning or reflective thinking, and 
is subject to the laws which psychologists have dis- 
covered to hold true for that form of mental effort. 

4, Kinds of information which may be required. 
There are so many different causes that might possi- 
bly create a maladjustment in a school child’s life 
that finding the cause is not always a simple matter. 
For that reason, a list of items to consider may often 
be of service in helping to make the search in an 
orderly and systematic way. The following facts have 
often been found to be important elements in shedding 
light on case studies and are commonly mentioned by 
writers on case study technique: 

(1) Physical condition. This includes not only such 
things as height, weight, age, maturity, etc., but also 
health, nutrition, bodily functioning, and physical 
defects. 

(2) Hereditary background. The roots of a given 
child’s difficulties may often be discovered in his par- 
ents or even grandparents. Knowing more about the 
family tree and the behavior characteristic of the 
parents may help to understand the child’s peculiari- 
ties. 

(3) Mental condition. A thorough investigation of 
a child’s general intelligence, reaction time, sensory 
discrimination, power of concentration, ability to 
study, and similar psychological characteristics, may 
direct attention to significant causes of existing mal- 
adjustments. 

(4) Emotional characteristics. This includes such 
things as nervousness, mannerisims, automotisms, 
epilepsy, ‘hysteria, daydreaming, lack of: persistence, 
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precocious sex development, unethical conduct, wan- 
derlust, impulsiveness, shyness, fears, bigotry, hobbies, 
attitudes, interests, motives—in short, the whole 
range of emotions attitudes, and instinctive tenden- 
cies with their modifications due to experience. 

(5) School record. This includes his record of pro- 
motion or non-promotion, marks earned on standard 
tests, teacher’s estimates, etc., as well as relative 
achievements in the different subjects, such as art, 
arithmetic, spelling, reading, etc. 

(6) Past history. This includes the record of acci- 
dents, shocks, disappointments, sickness, discord, un- 
happiness, jobs held or undertakings engaged in—in 
fact, a complete history of his experiences up to the 
present time. 

(7) Present activities. This includes such things as 
work done for money, recreation engaged in, pastimes, 
pursuits competing for his time, as well as any differ- 
ences observable between behavior at school and 
elsewhere. 

(8) Home conditions. This includes the number of 
parents living, number and age of brothers and sisters, 
financial status, occupations of parents, moral, social, 
and educational traits of his parents, living conditions, 
happiness or unhappiness pervading family relations, 
etc. 

(9) Social contacts. Under this head comes the 
neighborhood or community in which the child lives, 
the persons with whom he associates, the teachers or 
other adults with whom he comes into contact, the 
persons he likes or dislikes, and similar forms of 
social stimulation or lack of it. 

From the above outline, it will be seen that there is 
nothing in the child’s whole range of experience which 


CASE-STUDY TECHNIQUE 85 


may not conceivably have some relation to the problem 
which he presents. A case study, however, will not 
merely make an inventory of these things; it will 
rather examine each of these critically for its relation 
to the problem in hand. Information about these 
things is not to be accumulated for its own sake, or 
as an end in itself, but only as a means of solving 
the original problem presented by the child. 

5. Methods of securing information.. There are 
many different ways of getting access to needed in- 
formation for a case study. A few are as follows: 
(1) Observation of the child as he goes about his 
work may often be serviceable. Sometimes the only 
way to discover his difficulty is through such direct 
observation. (2) Interviewing the child, or having 
him report his own observations upon his behavior or 
his difficulties, is often necessary, although he is 
unable or unwilling in many cases to give an all- 
around account of his condition. Interviews at dif- 
ferent times with long intervals of time between may 
be compared to check accuracy of statements, as any 
person interviewed forgets what he told the last time, 
and, if he is making the story up, the two stories will 
not agree. (3) Interviewing parents, teachers, or 
others who have something to do with him will usually 
add new information. (4) Visiting the home for 
actual observation of the nature and character of his 
home life, or to see him in his natural setting, will 
often recall a side of his character or personality not 
observable in the school environment. This may also 
be a good preparation for interviewing the parents for 
their help in making a diagnosis. (5) Experimental 
procedures may at times be practicable; that is, con- 
ditions may be especially arranged beforehand and 
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varied at will to provide desired stimulations of the 
child for the purpose of observing his reactions. (6) 
At times it may be necessary to put the child into a 
special institution, such as a hospital, reform school, 
or institution for mentally abnormal persons, where 
the observations and analyses of his behavior may be 
carried out more thoroughly and also be continued 
over a longer period of time. In such cases it is some- 
times possible to get a longitudinal view of the case 
in addition to various cross-sectional views, following 
the development of the various phenomena in a way 
that would be impossible otherwise. (7) Rating scales 
and score cards of various kinds may be employed in 
judging subjective but nevertheless important mat- 
ters. (8) Tests may be administered to determine 
his intelligence, school achievement, or physical con- 
dition, and special tests may be devised for particular 
things. (9) The various methods of research which 
have been evolved by the psychologist may be put 
to service for this purpose in varying degrees. The 
chapter on psychological technique should be read as 
an extension and further elaboration of the present 
topic. 3 

6. Winning confidence and cooperation. Something 
can be accomplished in studying a child without his 
knowing that he is being studied. Interviews and ob- 
servations may be gained without his knowing that 
there is a purpose behind it all. As a general rule, 
however, the information secured in such ways is far 
from complete, and is very much in need of being 
supplemented by direct contributions from the child 
or from his relatives. If an attempt to secure infor- 
mation by disguised methods is discovered by the 
child or his parents, a feeling of suspicion or antag- 
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onism is produced which bars the way to the infor- 
mation that is desired. 

There are many kinds of information which only 
the child himself can supply, but which he naturally 
hesitates to supply. There is a much greater hesitancy 
about discussing mental weaknesses or abnormalities 
than physical ones, as if they involved a social dis- 
grace. It is even more difficult to get a frank discus- 
sion or confession regarding matters of morality or 
conduct. Matters concerning sex are spoken of very 
sparingly, if at all, and then only to persons of the 
same sex or to the physician. Frequently it is neces- 
sary to break a crust of suspicion or fear of the law 
and get this person to give up a secret which alone can 
solve the problem which his case presents. 

Some methods of accomplishing this are as follows: 
(1) Get the child to see that he is a problem to himself, 
and then kindly consent to help him solve his problem. 
(2) As an opening wedge, tell him you want to find 
out what his capacities and talents are so that you 
can help him fit himself into the world of vocations 
and other responsibilities where he can make the 
greatest success. (3) Assure him that what he says 
or confesses will in no way be used against him. (4) 
Keep what he reports in the strictest confidence, so 
that he will know that his personal characteristics and 
intimate revelations will not become matters of gen- 
eral knowledge or public record. (5) Appeal to par- 
ents and relatives from the standpoint of their parental 
love for and interest in their children. (6) Even when 
it is apparent that the information supplied is un- 
truthful or inaccurate, accept it as if it were not sus- 
pected, and check up on it from outside sources, rather 
than attempt to cross examine the person supplying it. 
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To manifest suspicion or disbelief simply encourages 
more lying. 

7. Records and case histories. There are several 
viewpoints from which to consider the matter of case 
records. (1) If schools and other institutions would 
keep complete, comprehensive, and cumulative records 
of all significant facts about all children with whom 
they deal, normal as well as exceptional, the difficulty 
of getting adequate information when children come 
to be problem cases would be greatly reduced. Syste- 
matic ways of preserving facts about intelligence, 
physical condition, and school achievement are coming 
to be more appreciated in recent years than was for- 
merly the case. (2) The investigator who conducts a 
case study should write the facts down in a reasonably 
complete and detailed form simply for the increased 
definiteness and precision of the study which the 
writing entails. Also the record, after it has been 
made, aids the investigator to study the case more 
comprehensively because it directs his attention to 
all aspects and prevents overlooking significant items. 
(8) Complete records should be made for the guidance 
of those who will have to deal with the child at a 
later date. Teachers, counsellors, research workers, 
etc., come and go, but the child demands continuity 
of attention and service. Unless the facts of a case 
are reduced to record, a new worker cannot pick up 
the case where the old one left off. Also, a case which 
was apparently cured or adjusted satisfactorily may 
suffer a relapse, in which circumstance it is advanta- 
geous to have the facts about the former treatment 
available. (4) Case records are of great service to 
other workers in diagnosing other cases of a some- 
what similar nature. In the early days of medical 
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practice, the few cases which were written up became 
the sources of guidance for other physicians, who only 
had sufficient skill to repeat the treatments which had 
proven successful. Social workers, court attaches, 
psychologists, physicians, and teachers could add much 
to the common body of science by putting their cases 
on record to build up an extensive body of data about 
human problems. (5) It has been suggested that 
case records should preserve the life and color of the 
real situation instead of containing statistical and 
factual statements. The style of the novelist would 
do more toward leaving a clear picture of the case 
than the catalog-entry style that has been commonly 
employed. 

8. Treatment. A case study ordinarily attempts 
more than simply to understand the case; it seeks to 
improve it as well. The ultimate result should be that 
the case ceases to be a problem. The peculiarity or 
abnormality which occasioned the investigation should 
be corrected and the person that was exceptional re- 
turned to the normal or typical group. The treatment 
growing out of the study should bring the individual 
back into line so that he will no longer be unique or 
exceptional. Treatment is therefore, a natural and 
normal part of the case study, just as treating a 
patient for physical illness is a natural part of the 
physician’s services. This phase of the work becomes 
a matter of service rather than research, and hence is 
outside the scope of this book. There are, however, a 
few suggestions on how to administer treatment which 
may be mentioned here. (1) Help the child to help 
himself, rather than assume the entire burden for 
him. (2) Give him a chance to weigh the consequences 
of cooperating or not cooperating in the treatment, 
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and to decide on the desirability of making the strug- 
gle. (3) Preserve his self-respect. (4) Keep up his 
hope. (5) Be severe at times if necessary, but make 
it clear that the severity is necessary to avoid even 
more unpleasant consequences that would be certain 
to result from lack of severity. (6) Modify the diag- 
nosis in the light of the outcome of the treatment, 
making the treatment serve as a final check or veri- 
fication of the accuracy of the diagnosis. (7) Do not 
be offended or feel hurt if the service rendered results 
only in personal antagonism or ingratitude. It fre- 
quently happens that the patient hates the physician 
who lances a boil for him, and that a professional 
servant who cures a moral or personal abnormality 
incurs the hatred of the person served. 

9. Generalizing. As was stated previously, the case 
study is concerned with what is unique or individual 
rather than with what is typical or universal. It rep- 
resents a radically different point of view from that 
of the statistical study which seeks to find the gen- 
eral and universal and ignores the unique or partic- 
ular. The two methods are, in a sense supplementary. 
To carry into case work the statistical frame of mind 
would result in almost certain failure. Case studies, 
by their intensive nature, must be limited in number, 
and are therefore unsafe as bases for sweeping gen- 
eralizations. And even if there were adequate num- 
bers, the cases would not be representative, because 
they exist only by virtue of their being exceptional. 
The range of variability of a number of such cases 
would be so great that any central tendency that might 
be observed would be only very roughly representative 
of the cases from which it was derived. Furthermore, 
any attempt to generalize or to classify cases into 
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types while investigating them might tend to warp 
the diagnosis to fit the type rather than to make it 
fit the individual case. 

And yet, in spite of all these negative considerations, 
there is a certain need for generalizing from case 
studies. It would be most extraordinary if a number 
of cases had nothing at all in common, or presented 
absolutely no similarities. The diagnosis and treat- 
ment of a dozen cases should be a good preparation 
for the diagnosis and treatment of another dozen. The 
ultimate result of case work should be the collecting 
of a large body of case records; our knowledge of 
exceptional cases would thus become so complete that 
these very exceptional cases would be reduced to 
general rules and cease to be exceptions. In the early 
days of medicine, smallpox was an exceptional disease, 
as were cholera and typhoid, but they are now known 
as common types because we have studied enough cases 
to be able to generalize about them. Similarly the 
educational problems that must now be dealt with as 
exceptions may finally be dealt with according to 
regular rules or laws. The point to note here is that 
the generalizing should not be mixed with the case 
study itself, but that it should come somewhat as a 
byproduct after a large number of case studies have 
been made for reasons of immediate practical necessity. 


EXERCISES AND QUESTIONS 


1. How does the case study compare with other types of 
research as regards its contributions to science? To 
practice? To the professional growth of the research 
worker? 


2. Why is diagnosis a central feature of the case study? 


3. What types of case studies seem to be required or needed 
in the school situation with which you are most familiar? 
How would you attack these? 
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Why must the case worker be something of a diplomat? 
What should case records contain? Why? 
May diagnosis and treatment be separated? Explain. 


What would be gained and what lost by refusing to gener- 
alize from case studies? 


8. How would case-studies in educational work differ from 
those made in medicine? In criminology? In social work? 


9. Make a list of the desirable or necessary characteristics of 
a good case worker. 


10. What special cautions should a case worker observe in 
order to keep his activities on a strictly scientific plane? 


11. Draw up an outline of the kinds of case studies which 
naturally fall in the scope of the classroom teacher. 
The principal. The school nurse. The attendance de- 
partment. 


So te 
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CHAPTER VI 
SURVEY TECHNIQUE 


General characteristics of a school survey. A school 
survey is in a sense a case study of a school or of some 
aspect of it, looking toward a diagnosis of its weak- 
nesses and the setting up of measures for improving 
them. Or it may also be thought of as simply a 
research program carried out for the improvement 
of conditions in a particular situation. A survey is 
not an aimless collection of facts, nor a collection of 
facts about all phases of a school system. It may 
be very specific in nature, covering only a single part 
of the school’s work in a very intensive way. It seeks 
to solve one or more definite problems. If no problems 
exist, there is no occasion for a survey. Surveys 
originate in the actual needs of the schools for prac- 
tical solutions of practical problems, rather than in 
the idle curiosity of the surveyors. Occasionally a 
survey may be of a very general sort, covering prac- 
tically every aspect of the school’s work, but even 
that type is for a definite purpose, or should be. More 
often the purpose is more specific, as in the case of a 
survey to determine whether a new building is needed, 
or whether a junior high school should be created as 
a part of the school system, or whether the achieve- 
ment in the common branches is as high as it should 
be. Even the general or comprehensive survey will 
ordinarily consist of a number of smaller parts, each 
of which, if done alone, would properly be considered 
a survey in itself. 
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Since the survey is an attempt to study a definite 
problem or set of problems in scientific ways, the pro- 
cedures to be employed are in general those which 
are applicable to other forms of research. The tech- 
niques which are selected must be adjusted to the 
particular situation and problem. Detailed plans 
should be made beforehand, and the whole investiga- 
tion organized with care, in the light of all available 
knowledge about scientific technique, to be sure that 
the results help to solve the problems which have 
occasioned the investigation. 

By way of summary of this topic we may say that 
the survey has three main characteristics; it is pur- 
poseful, practical, and confined to a particular situation. 


I. Steps Common to All Surveys 


Whatever the local situation or the particular prob- 
lem to be solved, each survey will have many things in 
common with all others. It is the purpose of this 
section to treat some of these general considerations 
which are common to all surveys, leaving to the next 
section certain more specialized matters. 

1. Selling the survey project to the community. 
Since the making of a survey is something out of the 
ordinary, and since it usually involves considerable 
expense, securing the support of the community is 
a matter of considerable importance. The general 
public frequently is quite unaware of the existence 
of the problem which is to be solved, or does not 
appreciate its importance. In such a case, the Super- 
intendent of Schools and the Board of Education must 
first be convinced, and then they must educate the 
community to appreciate the importance of the prob- 
lem, and then “sell” them the survey as a means of 
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reaching a sound solution. Experience has suggested 
several things that may be serviceable in this con- 
nection: (1) The amount of financial benefit that 
would result from the survey may be shown to coun- 
terbalance the small expense involved. (2) The im- 
portance of securing actual facts as a basis for action 
may be stressed. (8) The importance of safeguarding 
the truth by having competent and disinterested 
experts from the outside collect and interpret the facts 
may be used as a final and very effective means of 
influencing a decision. As a general rule, it is well 
to sell the idea to a few of the leaders, such as the 
board members and the members of the various ser- 
vice organizations, as a first step, and then to use the 
support: and sponsorship of these persons as an added 
means of winning a general feeling of cooperation 
throughout the community. Incidentally, it is neces- 
sary not only to sell the idea of the survey but also 
to sell the idea to a few of the leaders, such as the 
They must be shown to be competent to carry out the 
project. After these things have all been accomplished, 
the matter should be reduced to the status of a defi- 
nite contract between the school authorities and the 
survey directors so that the whole may be carried 
out legally and according to the plans which were 
agreed upon. 

2. Financing the survey. The cost of conducting 
a survey may be a considerable sum. A few items of 
expense are the tests used, the tabular forms for 
recording data, clerical service, special services of the 
draftsmen, map makers, etc., salaries of assistants, 
the fees paid to the experts who direct the work, cost 
of printing the report, etc. Several different methods 
of financing surveys have been employed: (1) In cer- 
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tain cases the support of some educational foundation 
can be secured, especially when the particular survey 
is such as will shed light upon the problems faced by 
a large number of other schools. The Russel Sage 
Foundation, The General Education Board, the Com- 
monwealth Fund, and the United States Bureau of 
Education have conducted surveys in certain cases 
at their own expense. (2) There are instances in 
which surveys have been conducted by the faculty and 
students of some graduate school of Education simply 
as laboratory training in the technique of school sur- 
veying, and without cost to the schools surveyed. (38) 
Not infrequently the survey expense is paid as a part 
of the disbursement for the building program which 
grows out of the survey. (4) The surveyor is some- 
times paid nothing for his services in making the sur- 
vey, but is employed as an educational expert to super- 
vise the carrying out of the program of remedial 
work which the survey finds to be needed. (5) Prob- 
ably the best plan, in most cases, is that of definite 
appropriation of funds from the school’s budget to 
pay the expense of the survey. 

3. Organizing the survey staff. A survey should be 
directed by one person. He should be in general con- 
trol over the entire undertaking and be responsible 
for keeping the entire project unified. The actual work 
of conducting the survey will, however, ordinarily be 
divided up into smaller units and be parcelled out 
among a number of assistants. One committee, for 
instance, may assemble data upon population growth 
while another is at work upon the matter of finances, 
and so on. Each of these committees should be under 
the. direction of a competent and responsible leader 
who is able to coordinate their efforts and to keep the 
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work going. These various committees will subdivide 
their respective tasks into still smaller units, so that 
each assistant will have personal responsibility for 
definite things as well as being cooperatively respon- 
sible for the group’s project of which his forms a 
part. The assistants who constitute these committees 
must, of course, be trained in their duties, either as 
a part of their previous graduate study or by special 
instruction in preparation for the actual survey, and 
their work should be checked over and supervised by 
the director as it goes along. In order to secure greater 
coordination among the different phases of the work, 
it is well to build up a social bond among the workers 
by taking lunch and engaging in recreation together, 
as well as by more formal meetings for discussion of 
the progress of the survey and the reports of the 
various committees. Incidentally, provision for suit- 
able rooms, work tables, calculating devices, and equip- 
ment for the actual doing of the work, is important 
both for its effect on the quality of the work and also 
for its aid in building a group consciousness among 
those who work together in the same place. 

4, Gaining local cooperation. Even though a survey 
is being conducted by outside experts, it can hardly 
succeed without the cooperation of the local school 
authorities, teaching staff, and community. Many 
essential facts are available only through such co- 
operation, and any obstructive tendency on the part 
of the local people is certain to handicap the work. 
For example, if the Superintendent and board attempt 
to put a survey program into effect without consulting 
the teachers, the latter are likely to feel that they are 
being investigated. Such a feeling may arouse sus- 
picion, prejudice, or even open hostility. The matter 
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of keeping everybody happy about the survey is of 
the greatest importance. Care must be exercised to 
avoid creating the impression that some one is going 
to be “shown up” or that the surveyors are engaged 
in a work of espionage. 

Not only is local cooperation valuable in removing 
barriers, but also as a means of getting assistance 
in the actual prosecution of the survey. Assistance 
given by teachers and administrative staff, instead of 
by imported assistants and clerical workers, not only 
reduces the cost of the survey but also raises the level 
of professional spirit and understanding on the part 
of those who give such assistance. Also the sugges- 
tions and criticisms of the school system which are 
given by the members of its staff may be very fruit- 
ful sources of information and of possible needs that 
are to be met. 

5. Securing needed information. The sources of 
data to be used in a survey will vary according to the 
specific problem to be solved, but a few general sug- 
gestions may be helpful: (1) Actual observation and 
visitation are essential; to attempt to conduct a long- 
distance survey is almost certain to result in missing 
many important essentials. (2) Legal or official doc- 
uments and records are generally to be preferred. 
These include the state constitution and statutes, the 
various county and city ordinances, the records of the 
county superintendent, the city or county treasurer, 
the tax assessor, city engineer, and similar officials. 
(3) The permanent records and minutes of the school 
board are of considerable importance as authentic 
sources of information about school regulations, poli- 
cies, and precedents, .as well as of the historical back- 
ground of the current situation. (4) The’ regular 


SURVEY TECHNIQUE 101 


school records and accounts which have been kept for 
previous years are an invaluable source of data for 
the surveyor. These will, if properly kept, tell much 
about teachers, pupils, finances, and similar matters. 
Often they can be made to yield valuable by-product 
information beyond that for which they were origin- 
ally intended. (5) The official reports of the Super- 
intendent of Schools may be used in many cases to 
get in nicely tabulated form what would require much 
labor to assemble in the first place. (6) Samples of 
all printed matter, textbooks, special bulletins, courses 
of study, mimeographed announcements, blank forms, 
and in some cases the actual files of correspondence, 
may be utilized to get information about the educa- 
tional program and matters of internal administration. 
(7) Special methods must frequently be devised to 
secure needed informtaion, including testing programs, 
questionnaires, interviews, and similar research tech- 
nique. 

6. Comparing with cther schocls. Comparison is a 
central feature of most school surveys. This is due 
to the fact that for many administrative procedures 
and problems there is a lack of absolute standards, 
thus leaving the practice of other schools as the prin- 
cipal criterion by which to judge an individual school. 
For example, if the particular school being studied 
has a much lower valuation of school buildings per 
capita of students than is the case for a large number 
of other cities of the same class, this is a strong 
indication, although not actual proof, that its building 
facilities are inadequate. Similar comparisons may be 
made on the basis of size of classes, teachers’ salaries, 
teacher-student ratio, percentage of daily attendance, 
scores on standard tests, organization of the admin- 
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istrative staff, ete. The data necessary for such com- 
parisons can be secured, in many cases, from official 
sources, in other cases from the reports of previous 
surveys, and in still others from published norms for 
the state or nation in connection with tests. There 
are a few suggestions which experience has shown to 
be important in connection with comparative studies: 
(1) It is necessary to be very certain that items com- 
pared are really comparable. For example, different 
methods of accounting may cause two cities which are 
really equal in a given thing to appear quite unequal. 
(2) Before comparisons are made, the data should be 
reduced to statistical form. Comparisons made in 
terms of adjectives are much less helpful than those 
expressed in numbers or scores. (3) When the items 
to be compared are relative to other items, as costs 
are relative to population, such ratios should be com- 
puted and used as a basis for comparison as will show 
up the essential facts without confusion with other 
elements in the situation. (4) The manner in which 
data are collected may greatly help or hinder the 
making of comparisons. if due consideration is made 
of the form in which data from other cities exist, 
those from the local school system may be assembled 
in a manner that is uniform with the others, and 
thus greatly reduce the labor of drawing comparisons. 
(5) If schools would develop more uniform systems of 
keeping records, such comparisons would be easier 
to make. 

7. Forming judgments and making recommenda- 
tions. The school surveyor is more than a manipulator 
of facts and statistics. He serves somewhat in the 
capacity of an expert adviser as well. He must sup- 
plement his figures in many ways by common sense 


SURVEY TECHNIQUE 103 


and expert judgment. He must not only find facts 
but interpret them. He must not only discover how 
efficient the schools are but make constructive pro- 
posals for their improvement. Once the actual facts 
have been secured and agreed upon, the remaining por- 
tion of the task is one of exercising judgment and dis- 
crimination to determine their practical significance. 
The recommendations or proposals that result should 
be clearly based upon the evidence, rather than drawn 
from personal opinion or propaganda. Two methods 
of increasing the reliability and soundness of expert 
judgments in school surveys may be mentioned here: 
(1) The use of rating scales, score cards, or check 
lists is serviceable, since it helps the mind to consider 
the situation in smaller units and in greater complete- 
ness of detail than would be possible otherwise. A 
building score card is an illustration of this. (2) Group 
discussion of matters requiring judgment may help to 
carry the process of weighing and judging various 
elements further than one person’s individual analysis 
would go. For this reason it would be well for the 
various members of a survey staff to meet at inter- 
vals for reports and conferences, and also for the final 
report to be submitted to a group of intelligent and 
interested persons, such as the school board, for final 
criticism before it is printed. 

8. Reporting the results. The results of a survey 
should ordinarily be published, not only for the sake 
of gaining public support and cooperation in putting 
the recommendations into effect, but also ag an added 
incentive to careful and thorough work in making the 
survey. The report should be written for the intelli- 
gent layman rather than for the Surveyor’s expert 
colleagues, and for that reason should be copiously 
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supplied with graphical representations of the facts 
which it contains. It should contain, either at the 
beginning or the end, a brief list of recommendations, 
stated in condensed form for ready reference, in order 
to provide something tangible for study and discussion 
by persons who could not or would not take the time 
to find those recommendations if they were buried 
in the body of the report. 

The publication of the report is closely associated 
with the campaign for bringing its results to the at- 
tention of the community and securing action upon 
the recommendations made. The persons making the 
survey are not necessarily obliged to direct the pub- 
licity campaign, but they are naturally expected to 
cooperate in it. The first step in this might well be 
the establishment of relations with the local press 
before the survey begins. Then news items may be 
published as the survey progresses, as the judgment 
of those in charge directs. There is, however, grave 
danger that such news items may appear prematurely 
and do more harm than good. Some writers on this 
subject prefer to hold all information until the survey 
is completed and open a regular campaign, others to 
keep the campaign going on the installment plan. 
There is general agreement that, so far as press notices 
are concerned, several articles, each on a separate idea, 
are preferable to a few comprehensive articles. An- 
other phase of the publicity campaign is that involved 
in making oral reports in person before the school 
board, service clubs, and other civic organizations. For 
this purpose, it is often well to prepare large charts, 
maps, and even lantern slides, which may be used to 
appeal in a visual way. 
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In the previous section of this chapter, we have 
considered certain general aspects of school surveys 
which are much the same for all types of surveys. 
It now remains to outline briefly a few of the out- 
standing features of some of the more common special 
kinds of surveys. As was stated before, a survey may 
be made in connection with almost any phase of school 
work, however large or small, but the present chapter 
will be limited to procedures involved in surveys of 
population, buildings, finance, administrative organ- 
ization, teachers, pupils, curriculum, and educational 
results. Further reading on each of these types may 
be done by consulting actual school surveys and more 
extensive works on survey technique. It should be 
made clear at this point that the various special types 
of surveys to be discussed here are closely related to 
each other, and that there is often much to be gained 
by conducting several types of surveys at the same 
time and in correlation with each other. For example, 
the surveys of finances and buildings have much 
to do with administration, curriculum, and teaching 
efficiency. 

1, Population. Surveys of the population of the 
community are sometimes of considerable importance 
in solving school problems. For example, the city 
may be drifting in a given direction, and a section 
which was formerly inhabited by native whites may 
be gradually becoming a foreign or colored settlement, 
calling for a different type of educational program. 
To discover the sociological or racial character of the 
population, and the directions of sociological changes, 
may call for the making of spot maps with a different 
color of spot for each home of a given nationality or 
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social status, or a spot for each transfer of pupils to 
or from the district, one color showing a gain in native 
whites, another color a gain in foreign-born, etc. 

Another population problem is the determination of 
the direction in which the city is moving, particularly 
for the light it may shed on the locating of new build- 
ings. The building permits for a period of years may 
be tabulated or located on a spot map to show up such 
trends, as may also records of the new installations 
of light, water, or gas meters or telephones, or the 
percentage of the total city votes cast by the different 
precincts. 

Again, the problem may be one of rate of growth, 
in addition to its direction, in which case the records 
of the various public utilities, the average daily attend- 
ance in school, the vital statistics showing birth and 
deaths, etc., may be used to supplement the data to 
be obtained from the United States Census. In mak- 
ing such studies these various indices of population 
growth should be used to check against each other and 
care should be exercised to avoid relying upon a very 
few or upon unstable indices. 

In more settled and static communities, where the 
direction and rate of growth may be of lesser import- 
ance, and the character of the population may be very 
little subject to change, the main problem is one of 
determining the present lccation of the pupils’ homes 
with reference to the schools. For such purposes, the 
simple device of placing one spot on a map to locate 
the residence of each pupil may aid greatly in analyz- 
ing the distribution of the pupils with respect to dis- 
tance from schools, relations to rivers, hills, railroad 
tracks and similar factors. 

2. Buildings. There are several items that may 
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enter into a building survey, a few of which will be 
mentioned here. 

(1) Rating the present buildings. This may involve a 
careful study of the present buildings, checking them 
against existing state building codes to determine 
whether they meet legal specifications for school 
buildings. More important than this, however, is the 
scoring of the buildings as to their suitability to the 
needs of the schools in the more professional sense. 
For this purpose, the Strayer-Englehardt score cards 
for school buildings might be used, or the buildings 
might be checked against any other commonly accepted 
and well recognized standards that are available as a 
means of measuring their suitability for school pur- 
poses. | 

(2) Determining present utilization of buildings. Not 
uncommonly a school administrator thinks he needs 
new buildings when in reality he has ample space for 
his needs but is not getting the use out of it. To 
determine the degree to which a building is being 
utilized, the number of classrooms vacant at each 
separate period in the day may be determined as a 
first step. Next might come a study of class size to 
determine what economies could be effected by putting 
small classes in small rooms and reducing the number 
of vacant seats. Or the total number of vacant period 
seats may be compared with the total peak enrollment, 
to determine the maximum economies which could pos- 
sibly be effected, or the total number of pupils that 
could possibly be accommodated by the present build- 
ings. Needless to say, it would be impossible to secure 
absolutely 100% seat utilization at all times. 

(8) Determining the amount of new buildings to be pro- 
vided. The principal steps in determining how much 
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new building space is needed are: (1) Finding out 
how completely the present buildings are filled or 
utilized, and (2) finding the rate of growth of the 
school population. Having these facts, it is simply 
a matter of subtracting the present capacity from the 
number of pupils to be provided for. 

(4) Locating new buildings. Finding where to locate 
a proposed new building is a matter primarily of 
population study, which has already been discussed. 
A few additional factors enter, however, such as an 
analysis of the geography of the region, particularly 
as concerns natural barriers, traffic risks, and health 
menaces. The economic phase of this problem also be- 
comes one of major importance at times, involving 
rather extensive investigations of land values in the 
various proposed locations. 

(5) Determining the character of new buildings. Each 
new building should be constructed with a definite pur- 
pose to house a definite educational project or to pro- 
vide for a particular type of educational program. The 
kind of curriculum to be offered must be considered in 
building the various shops, laboratories, etc., and may 
require a survey of the social needs of the community. 
This point will be discussed in a later paragraph. 

(6) Determining the ability of the community to provide 
the new buildings. Since new buildings are one of the 
very large items of school expense, no building survey 
can be entirely inseparable from a financial survey. 
This will be treated in a later paragraph. 

3. Finances. Any complete investigation of the 
financial side of a school system must necessarily be- 
come a task of considerable magnitude. Some of the 
elements that would enter into such a study are as 
follows: 
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~ (1) The legal basis of the school funds. This includes a 
study of the laws, constitutional provisions, city ordi- 
nances, court rulings, etc., governing the raising and 
handling of school money, and the relative importance 
of the different sources from which the funds are 
secured. This is closely related to the topic of admin- 
istrative organization, and will not be treated further 
here. 

(2) Community wealth. This involves analysis of data 
bearing upon property valuation, ratios between true 
and assessed value, amount of community income, 
amount of funds raised for all purposes by taxation, 
amount of increase in number and volume of savings 
accounts, wealth per capita, and so on. Such data are 
useful to shed light on the community’s capacity to 
support schools and are basic to many other questions. 

(3) Percentage of public funds devoted to schools.. A city 
may be very rich and able to support its schools, but 
actually be spending a very small proportion of its 
money for education. The surveyor might compare the 
distribution of the tax dollar in the local case with that 
of other cities, as a means of discovering how well the 
schools are being supported as compared with fire, 
water, park, police, and street paving departments. 

(4) Distribution of school costs. Next after a analysis 
of the tax dollar might well come a study of the school 
dollar. This would involve a comparison of the per- 
centage of all school funds going to new buildings, to 
interest on bonded indebtedness, to salaries of admin- 
istrative and teaching staff, to fuel, to supplies, ete. 
Such studies, if compared with similar ones from other 
cities having uniform systems of accounting, are likely 
to reveal any significant departures from standard 
practice in the distribution of expenditures and to sug- 
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gest desirable changes. Such comparisons may be made 
more objective if based on such items as cost per class- 
room, cost per cubic foot, cost per pupil class period, 
cost per teacher, cost per student in average daily at- 
tendance, etc. 

(5) Present balance sheet. This would involve some- 
thing of an analysis of all indebtedness, current de- 
ficits, surpluses, etc.—in short, a careful summary of 
present condition as regards finances. 

(6) Business management. This would include the or- 
ganization of the staff that has to do with expenditures, 
the method of determining what money is to be spent, 
the system of accounting employed, the making and 
operation of the budget, and all similar things having 
to do with the method of handling the funds. 

4, Organization and administration. A survey of the 
organization and administration of a school system may 
involve several different lines of investigation, such as 
the following: 

(1) Legal background. This includes a study of the 
state and local school laws and regulations, and the 
checking up on such matters as the jurisdictions of the 
various legally established authorities which touch 
upon the schools. For example, the discovery of the 
exact relations of the schools to the County Superin- 
tendent and the State Superintendent is essential to 
the best understanding of their organization and ad- 
ministration, as is also a knowledge of the methods by 
which school authorities are selected and to whom they 
owe allegiance. The school surveyor, working on the 
administrative system, will naturally need, therefore, 
to be a student of school law. 

(2) The general type of school organization. Some schools 
are organized on the basis of elementary, junior high 
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schools, and senior high schools; others have only ele- 
mentary and senior high schools. Some communities 
have all elementary schools under one head while the 
high schools are under different control. The particu- 
lar scheme of organization employed in the local situa- 
tion must necessarily be included as a part of the 
survey. 

(3) Composition of the school staff. The survey will 
ordinarily include an inventory of the various members 
of the staff, and comparisons of its composition with 
that of other cities of a similar type. Such comparisons 
might include such items as the ratio of number of 
non-teaching members to teaching members, number of 
supervisor to the number of teachers, etc. 

(4) Job analyses of the various members of the staff. 
Sometimes the problem of the survey is to redistribute 
the various administrative duties among the different 
members of the staff. The job analysis method is par- 
ticularly appropriate in this connection. It involves a 
more or less minute listing of the different activities 
or duties of each worker, as described more fully in 
the chapter on job analysis technique. 

(5) Organization charts. The steps previously men- 
tioned will ordinarily lead toward the making of an 
organization chart. This is a graphic picture of the 
administrative staff, with a square to represent each 
member and lines joining these various squares to show 
lines of authority and of responsibility. If these charts 
are carefully made and truly represent the functions 
that are being performed by the persons whose work 
is charted, they become very effective means of dis- 
covering needed changes. 

(6). Relations of existing status to principles of school ad- 
ministration. When the facts about the actual adminis- 
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trative situation have been discovered, it should not 
be very difficult to check them against sound principles 
of school administration to determine wherein the prac- 
tices are unsatisfactory. For example, such a check 
might reveal that a certain school hag a dual control, 
with a business manager and a superintendent of 
schools on equal footing, each independent of the other 
and responsible directly to the board, an arrangement 
which violates the principle that there should be 
unified control under the superintendent. 

5. Teachers. A survey of the teaching staff might 
undertake to discover several different things, among 
which the following are representative: 

(1) The sources from which the teachers come. This 
would include such factors as percentage of home-town 
teachers contrasted with new blood from outside, as 
well as the sociological character of the homes from 
which the teachers are drawn. 

(2) The machinery for electing teachers. This might in- 
volve an analysis of the methods used to discover good 
teachers. The standards used in selecting the best 
from a number of applicants, the qualifications of the 
persons making the selections, and all other factors 
which shed any light upon the extent to which the 
methods employed are conducive to the securing of an 
efficient corps of teachers. 

(3) Tenure. Studies of tenure might cokes data 
upon the average length of service of the teachers, the 
annual rate of turnover, and the factors which affect 
these. A low turnover might be due to unusually high 
salary scales, or to the fact that the teachers are of 
a lower grade and contented to remain because they 
are not disturbed by. efforts to secure protessiai. 
growth. : 
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(4) Character and composition. Under this head would 
come such questions as the percentage of women teach- 
ers, the nature and amount of academic and profes- 
sional training, the amount and character of teaching 
experience, age, race, etc. 

(5) Salaries. Frequency tables showing the range 
and distribution of salaries, classified according to 
amount of training and experience and also according 
to nature of the teaching positions held, should shed 
considerable light upon the teacher situation, especially 
if such distributions are compared with similar ones 
from other cities. 

(6) Teaching loads. Studies of teaching load not only 
include the number of hours of teaching per week, but 
also the character of the service rendered. They should 
also include investigations of the relation between sub- 
jects taught and the kind of previous preparation 
which was made for teaching, as well as the amount 
and character of extra-curricular and administrative 
duties that are performed in addition to teaching. 

(7) Efficiency of instruction. The quality of service 
rendered by the teachers may be studied from either 
of two angles, process or result. Actual visits to the 
classrooms may reveal much about teaching efficiency, 
as may also results of tests which are given to the 
pupils. This point will be treated in a later paragraph. 

6. Pupils’ progress. A very common and very im- 
portant part of any complete school survey has to do 
with the progress of pupils through the grades. The 
several ways in which this may be analyzed are illus- 
trated by the following: 

(1) Elimination. Some idea of the number of students 
who are eliminated year by year may be gained by 
comparing the total enrollments for different grades. 
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These studies usually show a marked decrease of stu- 
dents in school after the sixth or seventh year. Care 
should be exercised, however, to avoid attributing the 
effects of retardation to elimination. 7 

(2) Retardation and acceleration. Data upon the num- 
ber of students who are repeating grades, or who for 
other reasons are below the grades normal for their 
ages may shed considerable light upon the effectiveness 
of the schools. It may indicate that too rigid a stand- 
ard of work is being maintained, or that the instruction 
is inefficient, or any one of several other things. An- 
alyses of the data on acceleration would naturally ex- 
tend and supplement studies of retardation. Age-grade 
tables are very useful for these types of studies. 

(8) Classification. The effectiveness with which chil- 
dren are grouped into classes for instruction may be 
determined in some measure by consulting the records 
to discover the extent of homogeneity within classes. 
It may be found, for example, that students are being 
forced to work together who are not homogeneous on 
any basis, or that in the effort to secure homogeneity 
as to scholastic achievement pupils of widely different 
ages, sizes, and intelligence quotients have been 
grouped together. The simple device of making a 
group photograph for each grade or class may help to 
show up in a visual way the extent to which there is 
homogeneity as to size and physical maturity. 

7. Curriculum. Surveys of the curriculum may in- 
volve some of the research techniques and principles 
which have been treated in the chapter on curriculum- 
making technique. In addition to these, there are cer- 
tain other considerations which may be mentioned 
here: 

(1) The population survey may shed considerable 
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light upon the kind of curriculum that is demanded for 
the particular community. This may take the form of 
an occupational or vocational survey of the community, 
listing the different kinds of occupations and the an- 
nual supply of new workers required in each. Such 
studies constituted an important part of the well- 
known survey of the Cleveland public schools. 

(2) The nature of the present offering may be dis- 
covered by a critical examination and tabulation of the 
content of textbooks, printed courses of study, or rep- 
resentative samplings of the written work done by 
pupils in the different classes. 

(8) The costs of the various subjects in the curricu- 
lum can be determined by comparing such items as 
teachers’ salaries, teaching loads, class size, building 
space used, equipment required, etc., resulting in defin- 
ite figures indicating cost per pupil class hour in the 
different subjects. Such studies have revealed great 
discrepancies between the costs of different subjects 
or the same subject in different schools, and have led 
to worth-while improvements in the nature of the cur- 
riculum offerings. 

8. Educational results. It has been said that “the 
proof of the pudding is in the eating.” Likewise, the 
-efficiency of a school system must be determined ulti- 
mately in the light of results secured. Measurement, 
or testing, naturally constitutes the principal device for 
determining the results. The complete school survey 
will ordinarily include a testing program. For this 
purpose standardized tests are necessary in order that 
the results achieved in the local situation may be com- 
pared with established norms from other schools or 
cities. In conducting such a testing program the labor 
of testing every child in every grade may be so great 
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as to make the task impossible. A sampling procedure 
is the device to remedy this difficulty. As a general 
rule, two types of samplings should be made. One sam- 
pling might consist of a measurement of every child in 
every subject in a given grade, thus getting a complete 
picture of the sixth grade, for example. The other 
sampling might consist of measurements of a certain 
subject, such as arithmetic, in every grade. Thus a 
horizontal sampling supplemented by a vertical samp- 
ling, might tell more about results of instruction than 
two or more samplings of one type only. Further dis- 
cussion of this aspect of surveys may be obtained by 
referring to the chapter on measurement technique. 


EXERCISES AND QUESTIONS 


1. How does the educational survey resemble and how does it 
differ from the social survey? The industrial survey? 
The engineering survey? é 


2. What part has the school survey played in the development 
of educational science? How does its service in this 
respect compare with the contributions made by other 
research techniques? 


3. What are the essential qualifications of one who would make 
a school survey? Justify each item, 


4, To what extent is it legitimate for a survey to be propa- 
ganda? 


5. How should a survey report differ from other research 
reports? How resemble them? 


6. What is to be gained by making a comprehensive or complete 
school survey as compared with a special-type or limited- 
scope survey? What is lost? 


7. List the problems in the school system with which you are 
most familiar which seem to call for a school survey. 
How would you go about attacking each? 


8. Referring to the special types of surveys mentioned in this 
chapter, show what each type might contribute to the 
others, or how it might be correlated with them. 
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the school survey. 
Trow, Wm. C. Scientific Method in Education. Houghton 
Miffin Company, Boston, 1925. 
Pp. 118-123, “The Social Survey,” pp. 123-124, “The In- 
dustrial Survey.” 


The reading of actual survey reports is one of the best ways to 
get acquainted with survey technique. 


CHAPTER VII 
CURRICULUM MAKING TECHNIQUE 


1. Practical versus scientific curriculum making. 
There are two types of curriculum work under way at 
the present time, both of which are entirely justifiable 
and worthy. One is the scientific, or strictly research 
type, looking toward getting actual facts about what 
should be taught. It seldom attempts more than one 
small and specific problem at a time and attempts to 
do that one thing in a thorough and conclusive way. 
The other type is the practical type involved in course- 
of-study making. It is prompted by the immediate 
necessity of doing something now and can not wait for 
the slow and laborious work of the scientist. It involves 
the application of common sense and the judgments of 
competent persons in selecting and arranging materials 
for instructional purposes tentatively, with the idea 
that it is better to have what experts think is a good 
curriculum than to have what has already proven by 
experience to be unsatisfactory. This is entirely sound 
reasoning. It is better to have an empirical foundation 
for educational practice than no foundation at all, and, 
lacking a strictly scientific foundation, an empirical one 
based on common sense and the observations of educa- 
tional specialists is about as good as we can have. 
Without at all belittling this practical type of curri- 
culum-making, it is our purpose in this chapter to treat 
the methods and techniques for the other, or more 
strictly scientific, type of work. The practical type of 
curriculum-making is quite commonly done by regular 
classroom teachers and school officials, and ordinarily 
begins with the classroom situation. The scientific 
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type is more often done by research workers or experts 
somewhat further removed from the actual firing line, 
and with the use of somewhat more specialized research 
techniques. It is hoped that more extensive training 
in such techniques may accelerate this very important 
type of scientific work. 


2. Importance of fundamental principles. An import- 
ant first step in doing scientific work on the curriculum 
is to master the fundamental philosophy of education. 
Briggs has expressed this point by the statement that, 
unless we get down to the underlying principles, our 
work will be of a superficial sort, or “curriculum-tinker- 
ing” rather than curriculum-making.* 

The basal assumptions upon which the work is 
founded must be sound or the work is not valid. The 
curriculum maker must be a wise and sensitive inter- 
preter of the spirit of the age, being a close student 
both of society and of education. He must not put too 
much faith in purely mechanical techniques. It is pos- 
sible to analyze the human body into its chemical con- 
stituents, but the element of life in it is not so readily 
subject to analysis. Likewise the curriculum-maker 
may analyze society or human activities in an objective 
way and still go wide of the mark in the resulting curri- 
culum because of failing to consider the purpose or 
spirit that moves through it all. To be more concrete, 
the research worker should at least spend considerable 
time in reading books on the curriculum by such men 
as Charters, Bobbitt, Snedden, Bonser, Meriam, Clem- 
ent, Briggs, etc. 

3. Activity analysis. Probably the most fundamental 
technique in curriculum research is the activity analy- 


*Briggs, Thomas H., “Curriculum Problems,” The Macmil- 
lan Co., New York, 1926, Chapter I. 
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sis, sometimes called job analysis. It is based on the 
sociological principle that education should prepare for 
the activities in which people engage. Having discov- 
ered what are the various kinds or types of human 
action, discovery of the kinds of training needed fol- 
lows quite naturally. The advantage of this procedure 
is that it is less influenced by personal opinion. A cur- 
riculum based on an analysis of desirable human traits 
or qualities involves considerable exercise of personal 
judgment in bridging the gap between the traits to be 
developed and the kind of training required to develop 
them. A good illustration of an activity analysis used 
for curriculum research is Lehman’s study of “Play 
Activities of Persons of Different Ages.’* He deter- 
mined in considerable detail what activities people en- 
gage in “just for fun,” and was able to derive from his 
analysis a number of very significant conclusions bear- 
ing on the curriculum. 

4, Difficuity analysis. A special form of the activity 
analysis is that which considers only those activities 
which are performed imperfectly, or which offer diffi- 
culty. Analysis of errors is often less laborious than 
analysis of activities, and is often a better guide in 
curriculum-making, because it is not necessary to teach 
the things which are already done correctly. The diffi- 
culty analysis, therefore, consists of finding where 
practice breaks down or falls short, and discovering 
from these deficiencies the kinds of training needed to 
correct them. A difficulty analysis is often useful as 
a means of completing an activity analysis, and vice 
versa, since what is overlooked under one classification 
is likely to show up more visibly under the other. A 


*In Bobbitt’s “Curriculum Investigations,’ University of 
Chicago Press, 1926, Chapter XIV. 
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good illustration of a difficulty analysis is the study of 
grammatical errors of children in Kansas City made by 
Charters and Miller,** in which all mistakes in English 
were recorded and classified as a basis for determining 
the kind of grammatical training required by children. 

5. Securing objectivity. In making an activity analy- 
sis or a difficulty analysis it is preferable to do it in an 
objective way rather than to rely on personal estimates, 
generalizations, or opinions. Bobbitt’s work at Los 
Angeles has been criticized because of its somewhat 
subjective nature. He had many people think of all 
the objectives they could under such heads as health, 
mental efficiency, etc., and then made a complete list 
of all these items. These were naturally given from 
memory and in the lump rather than from direct ob- 
servation of actual human activity. If each person’s 
contributions had been based on analysis of a particu- 
lar person rather than ‘‘people-in-general”, the compos- 
ite would have been a result of analysis rather than 
one of analysis and generalization mixed in unknown 
proportions. It should be said, in justification of Bob- 
bitt’s procedure, that it was of the practical rather 
than the strictly scientific type of curriculum making, 
and was the only one feasible under the circumstances. 
Methods of securing objectivity in the analysis are not 
readily available, especially in analyzing mental rather 
than physical activities. Yet the need of objectivity is 
great, and the careful worker will do his best to invent 
special tools or instruments of precision for the pur- 
pose. A few illustrations of such devices may be men- 
tioned here. (1) Frequency of error, for example, in 
spelling. (2) Keeping of diaries by the individuals 
whose activities are being analyzed. (38) The case 


**Charters, W. W., “Curriculum Construction,” The Macmil- 
lan Co., New York, 1923, Chapter XVII. 
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method, as used by the sociologist and clinical psycholo- 
gist. 

6. Dividing the field. With the facilities available 
at present it is practically useless to attack the curri- 
culum problem as a whole in a single investigation. It 
is too large to be handled as a unit. Such an investiga- 
tion involves a complete and comprehensive analysis of 
the activities or difficulties of millions of people in thou- 
sands of life situations, and consequently, needs to be 
divided up into smaller units worked on separately. 
For example, one certain occupational pursuit may be 
analyzed as a basis for vocational training in that one 
line, or all spelling activities may be analyzed, without 
considering language activities or mathematical be- 
havior. The great essential in doing this is, however, 
that the part must not be mistaken for the whole, and 
that the results of a fragmentary analysis must not 
be offered as a solution of the curriculum problem in 
general. For example, an analysis of occupational 
activities must not lead to the omission of training for 
leisure, or non-vocational activities, nor should the oc- 
cupational activities be overlooked because the basis 
of analysis happened to be academic in nature. Some 
illustrations of investigations dealing with special 
phases of the curriculum problem are found in Bobbit’s 
Curriculum Investigations and in Charters’ Curriculum 
Construction, and include studies of such units as labor 
problems, etiquette, mathematics, citizenship, pattern 
making, etc. 

7. Short cuts. The task of making an analysis of 
activities directly is sometimes so great as to be prac- 
tically impossible. In such cases it is necessary to 
abbreviate it by some kind of lavor-saving device. The 
method of sampling is one such device. This involves 
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selecting some situation or field of human activity that 
is known to be fairly representative of society in gen- 
eral and making an analysis of it, considering the re- 
sults as equivalent to what a more extended analysis 
of the whole would have produced. This is entirely 
justified, provided the sampling is of adequate size and 
is known to be representative. Some examples of this 
procedure are given in Bobbitt’s Curriculum Investiga- 
tions, Where analyses of “major fields of human con- 
cern” were based on such samplings as the Encyclopedia 
Britannica, the Literary Digest, the Reader’s Guide, news- 
papers, the Sears Roebuck Catalog, the dictionary, 
political party platforms, etc. Any such short cut is 
only as valid, of course, as its underlying assumption; 
namely, that the sampling used is a true representation 
of human life. 

8. Limits of analysis. A curriculum study may err 
either on the side of too much or too little analysis. 
The usual error is on the side of too little. It has not 
been uncommon for an educational philosopher to an- 
alyze life activities into four or five divisions and pro- 
ceed to build a curriculum upon those divisions. Obvi- 
ously such a procedure is questionable because the 
units are too large to be grouped adequately. A much 
safer and more dependable basis would be an analysis 
of life activities into 10,000 or 20,000 smaller units 
which could be classified and grouped into a number of 
larger divisions. The labor involved in this task ex- 
plains why most curriculum work suffers from insuffi- 
cient rather than too much analysis, but it is a part of 
the price of curriculum progress. Charters says that 
the perfect curriculum is one that is analyzed down to 
the point that it is self-teaching. If it is in text-book 
form it needs no further supplementing or explaining 
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by the teacher. He says that “‘neatness”, “initiative’’, 
and “ability to think” are too large units, and should 
be subdivided into such specific elements as ‘“‘brushing 
your coat” and “combing your hair.” In other words, 
he would carry the analysis to that point which most 
nearly conforms to the ‘‘mental stride” of the learner, 
taking pains not to present material in units too large 
to be comprehended, nor too small to yield interest or 
perspective. 

9. Progress. In making an analysis for the curri- 
culum it is necessary to choose the best or most successful 
rather than the average as a basis upon which to work. 
For example, a teacher-training curriculum should 
train persons to teach according to the practice of good 
teachers rather than that of ordinary teachers. If the 
curriculum is derived from the practices of the highest 
stratum of society, the result is the raising of the entire 
next generation to that level. This is society’s method 
of raising itself “by its own bootstraps.” The curricu- 
lum should. educate for life as it ought to be lived 
rather than for life as it is. Analysis of the activities 
of the ordinary or the unsuccessful strata of society 
may be fruitful in yielding difficulties or deficences not 
so readily seen in the most successful groups, but the 
higher strata are likely to offer the most effective 
methods of solving these difficulties. In fact, a very 
fruitful attack upon the curriculum problem igs to con- 
trast the methods or activities of the most successful 
people with those of the least successful and to train 
the next generation to follow the example of the most 
successful. 

10. New fields. A much-needed type of research is 
that of collecting and organizing curriculum material 
for new fields that have never been taught before. 
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Without it, the addition of new subjects to the curri- 
culum is impossible. As society advances, new prob- 
lems arise and new difficulties are faced which call for 
training that was not needed in an earlier day. For 
example, new social and industrial conditions have pro- 
duced changes which are seriously threatening the 
structure and life of the home. This presents a chal- 
lenge for the curriculum maker to assemble material 
or pupil experiences which will prepare for home mem- 
bership. The technique for work in pioneer fields of 
this kind is the same in general as that for older fields, 
but differs somewhat in degree. As a general rule such 
a field offers a better opportunity for truly scientific 
work than an older one, because there is less of the 
older structure to get rid of while building the new. 
For details of technique for this type of work the 
reader is referred to the chapters on job analysis tech- 
nique and interview technique. 

11. Emotional factors. A curriculum is a failure un- 
less it produces actual improvements in conduct. To 
tell how to do a given act is not effective unless the 
telling actually results in better doing of that act. 
Facts are almost helpless. in the face of emotions, but 
when properly charged with emotions, they become 
dynamic agents. The curriculum maker will not only 
endeavor to determine the activities which should be 
learned but also the ideals or emotional factors which 
should motivate and direct their performance. It is 
useless to teach how to be honest unless we succeed also 
in teaching to be honest. The curriculum must moti- 
vate the desirable types of behavior, and must there- 
fore include reasons for performing them in addition 
to directions about how to perform them. Charters 
has stressed very much the importance of a two-dimen- 
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sional type of research on the curriculum, activities 
being one dimension and ideals the other. Briggs has 
given approximately a third of the space in his 
Curriculum Problems to a chapter on “Emotionalized At- 
titudes.” The curriculum maker must not forget that 
the child is more than a mere intellectual being, and 
that training must reach the emotions as well as the 
mind and the hand. 

12. Evaluating objectives. After a reasonably com- 
plete list of proposed or tentative objectives has been 
arrived at by analysis or otherwise, the curriculum 
maker is faced by the problem of evaluating them and 
selecting the most important ones. Frequency of men- 
tion is not a perfect basis for evaluation, even though 
it is a fairly suggestive one. Importance is to be con- 
sidered as well as frequency. For example, some activ- 
ities are engaged in only once or twice in a lifetime, 
such as getting married, and yet they are nevertheless 
of very great importance. Contrasted with this, we 
find such activities as tying one’s shoes, which are per- 
formed every day but are relatively unimportant. In 
the absence of better methods it is often necessary to 
resort to the judgments of experts to evaluate object- 
ives. Bobbitt submitted to the teachers of Los Angeles 
long lists of objectives which had been collected by his 
graduate students and let them approve or disapprove, 
add or modify according to their judgments of the 
needs of the particular locality. When such methods 
are employed, they should be used with the conscious- 
ness that the opinion of a majority does not necessar- 
ily establish a thing as the truth, and that the judg- 
ments of a few experts may be more reliable than those 
of many persons of lesser training. 

13. Deriving pupil experiences from objectives. 
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Charters has called attention to the fact that a good 
aim does not necessarily issue into a good curriculum. 
Having set up the objectives that are to be achieved, 
the next problem is to find means of achieving them. 
These means are the pupil experiences, or subject mat- 
ter, of the curriculum. Extreme caution is necessary 
to avoid jumping from objectives to subject matter 
without any assurance that the subject matter chosen 
actually achieves the proposed objectives. As a general 
rule some form of personal judgment or concensus of 
opinion will be necessary in performing this operation. 
Bobbitt had the teachers select the pupil experiences 
which they thought would best achieve the objectives 
agreed upon for their subjects. 

14. Evolution versus revolution. The curriculum 
maker faces the same problem as every agent of social 
progress, whether to build upon what already exists or 
to discard it and start anew. The evolutionary plan 
would be to begin with the school subjects which we 
now have and make additions, substitutions, and elim- 
inations, hoping ultimately to make it fit social needs. 
This is more desirable from the standpoint of tempor- 
ary expediency, as in the case of the practical course- 
of-study maker, since it is more likely to result in 
something that can be put to immediate use, but it is 
not very desirable for strictly scientific work. The sci- 
entist will ordinarily find it preferable to forget the 
traditional moulds in which the curriculum is cast 
and to base his study upon life as it is actually lived. 
If the analysis of society finally results in such subject 
divisions as we now have, he will accept them, but if 
not he will not hesitate to discard them. Curriculum 
work by the evolutionary plan involves staying so close 
to the present practice that it is difficult to avoid falling 
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into the old established grooves, while the revolution- 
ary plan enables the investigator to get farther away 
and work without this handicap. The revolutionary 
method has, therefore, far greater possibilities of prog- 
ress in it than the evolutionary. 

15. Prevailing practice. A very large amount of 
effort is going into curriculum studies which aim to 
discover what others are doing. This type of curricu- 
lum-making has been called the “Scissors-and-paste” 
method, since it ordinarily attempts to find out what to 
teach by imitating the example of others. It usually 
means that the blind are leading the blind, and it is 
not unusual if the imitators fall into the ditch along 
with those whom.they imitate. It is based on the age- 
old fallacy that the majority is sure to be right. And 
yet this type of work has possibilities of real service 
if used with discretion. For example, if a number of 
schools are selected because of their demonstrated 
superiority of educational output and another group is 
selected because of their well-known failure to make 
successful members of society, then a study of the dif- 
ferences between the curricula of the two sets of 
schools might have some practical and scientific value. 
The wholesale and promiscous tabulation of prevailing 
practice, and the generalization that whatever is is 
right, however, is to be most emphatically condemned. 

16. Psychological factors. The main emphasis in this 
chapter has been placed upon the sociological determin- 
ers of the curriculum. The psychological factors, how- 
ever, should not be overlooked. The curriculum should 
be based on two main considerations, the activities of 
society and the nature of the individual learner. Chil- 
dren’s interests have much to do with what is to be 
taught, as well as its order and arrangement. A most 
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fundamental type of curriculum research that is urg- 
ently needed is the discovery of the facts about child 
nature, particularly as it relates to the learning of the 
various types of subject matter. The psychology of 
school subjects is an essential unit in any complete 
program of curriculum making. 

17. Child versus adult. In making an analysis of 
activities or difficulties for curriculum purposes, it is 
necessary to put the child in the foreground rather 
than the adult. To analyze adult civic duties ever so 
completely would not yield a suitable curriculum for 
citizenship to be taught to a child, because the child’s 
civic duties are different. To teach a purely adult 
activity to a child requires that the training lie idle 
for a perod of years until the child reaches maturity. 
During this time it is forgotten and becomes of little 
or no value. A better plan is to find out what children 
do now and train them to do those things better, and 
thus plant seeds that will grow to fruitage in actual 
conduct as they get older. It is unwise to attempt to 
prepare for adult life by cold storage education during 
childhood. 

18. Individual differences. Curriculum making would 
be simple indeed if all people were only alike! Indi- 
vidual differences complicate the problem and make its 
solution almost impossible. The activities of one per- 
son are different from those of another, making neces- 
sary a different kind of education. To take account of 
this, it is necessary to classify the persons who are in- 
cluded in the analysis into their various types so that 
each type is somewhat uniform and then to draw up 
for each type the kird of curriculum it requires. The 
same thing is necessary also on account of differences 
in intelligence, physical strength, sex, etc., making it 
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necessary to provide different kinds of training even in 
preparation for the same life activity. The curriculum 
maker can never hope to complete these various needed 
adaptations. He will simply carry his work to a point 
where the classroom teacher can take it up, and leave 
to her the final and more detailed adaptations to the 
needs and personalities of her individual pupils. 

19. Grade placement. Much remains to be done 
even after the subject matter for the curriculum has 
been selected. It must be arranged in the proper order 
or sequence to give the training needed at each period 
in the life of the child. Certain items of knowledge 
that have great value at the time of puberty may be 
worthless if postponed too late, and worse than worth- 
less if presented too early. The difficulty of subject 
matter must also be determined in order to avoid con- 
fronting the child with tasks which are too easy or 
too hard. Studies of elimination also have to be made 
before grade placement of materials can be definitely 
solved, since it is important to impart certain infor- 
mation even when it is premature accordng to psy- 
chological standards, if the probability is very great 
that the child will quit school before he would other- 
wise get that essential information. 

20. Propaedeutic functions.. The question of the 
child’s previous preparation for an advanced course is 
an ever-recurring one. College students are said to be 
deficient in grammatical preparation for foreign lan- 
guages or in mathematical preparation for engineering, 
etc. In some of these instances the deficiency is real, 
in others only imaginary. It would be of considerable 
practical value for the present day, and possibly also 
for the ultimate advancement of curriculum science, 
if more studies were made to determine exactly what 
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the prerequisites in knowledge, skill, and attitudes for 
the various subjects really are. In one sense, however, 
such studies might be considered as merely “curricu- 
lum tinkering” because they would be of no value if 
a more fundamental type of research should lead to 
the abolition of the present list of school subjects and 
the recasting of the whole program according to 
different moulds. 

21. Standards of perfection. In the case of some 
units of subject matter the problem is not “what” 
but “how much.” For example, we are all agreed that 
handwriting and addition should be taught, but 
to what degree of skill? Certain early studies in this 
line attempted to solve the problem by asking suc- 
cessful men how much of certain abilities they thought 
were needed. More recently the device of finding out 
how much of these abilities they actually possess has 
been employed. For example, Washburne actually gave 
standardized arithmetic tests to outstandingly success- 
ful business men to find how well they could add, sub- 
tract, etc., and from the results drew conclusions re- 
garding the amounts of these various abilities which 
are required for success. Standards arrived at by such 
methods should be generally preferred to those arrived 
at by personal opinion, even of experts. 

22. Other educational agencies. The school does not 
have a monopoly of educational service. Home, church, 
playground, newspaper, etc., all play their various 
parts. The curriculum maker must not assume that 
his list of objectives for education are the objectives 
for the school. Which objectives to assume for the 
school and which to leave for other agencies to achieve 
is a problem calling for special research. Special 
investigations of the educational services rendered by 
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the Boy Scouts or the Y. M. C. A., for example, fall 
legitimately in the field of curriculum research. 

23. Measurement. After a curriculum is made and 
installed, it should be evaluated by some kind of meas- 
urement. This can be done in a number of ways. One 
way is to try it out for a semester or a year and 
decide whether it works. This is usually done by 
merely subjective estimates of the teachers and other 
school officials. Quantitative methods are preferable. 
For example, the abilities of the children may be 
measured by objective tests before a new course is 
installed and again afterward to record change. The 
value of the course should be measured by its effects 
on the children rather than by the impression it makes 
on the teacher. For exact measurement it is often 
well to employ the experimental method, using two 
or more different courses, with as many parallel and 
equivalent groups, to determine their relative merits. 
Another plan is to administer standardized diagnostic 
tests after the installation of the new curriculum and 
observe the sections upon which the children make low 
scores. In other words, this method seeks to find the 
weak points and to apply additional training to fill the 
gaps. Well-kept age-grade tables would shed some 
light on the curriculum since undue amounts of retard- 
ation or acceleration at certain points would indicate 
how well the course was suited to the children. In 
closing, it is suggested that if the failures as well as 
the successes in curriculum-making were put into the 
records of society through publication much wasted 
effort would be prevented. Without this the same ex- 
periment may be tried out independently in several 
different places and abandoned in each case without 
ever a warning to others that it will not work. 
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EXERCISES AND QUESTIONS 


1. Why is curriculum research one of the most difficult types 
of research? 


2. Justify the emphasis which scientific curriculum making 
places on analysis. 


8. List the factors or variables that should influence the cur- 
riculum. 


4, Debate the relative importance of the child and of society 
as curriculum determiners. 


5. Analyze the school situation with which you are most 
familiar and list as many curriculum problems for re- 
search as you can. Outline briefly a method by which 
each might be attacked. 


6. Read the viewpoints of several writers on curriculum mak- 
ing and classify them according to their proposed methods 
of attack. 


7. Read a report of a piece of curriculum research and criti- 
cize the technique employed. 


8. What difference in research technique would there be be- 
tween research on an old school subject, like arithmetic 
or reading, and a new one, like thrift or safety? 
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CHAPTER VIII 
JOB ANALYSIS TECHNIQUE 


The procedure of job analysis is a very recent de- 
velopment, and one which has not yet gained as general 
recognition and acceptance by research workers as has 
been accorded to such older procedures as the experi- 
mental, historical, or statistical. In view of this gen- 
eral unfamiliarity with it, the following paragraphs 
of this section attempt to clarify its nature and pur- 
poses, after which other sections will deal more in- 
tensively with its technique. 


I. Nature and Purpose of Job Analysis 


1. Increased objectivity. Good workmanship in any 
line is to a large extent a question of personal opinion 
until it is analyzed into its elements. For example, a 
supervisor cannot judge teaching efficiency in the 
lump, except in a most subjective way, but two super- 
visors can arrive at very nearly the same estimate 
of a teacher if they use a rating scale which considers 
the various elements in teaching efficiency. When any 
complex object or process is studied in the lump, only 
crude judgments can be made, but if the various com- 
ponent elements are separated and studied indepen- 
dently, it may be possible to be entirely objective and 
scientific in dealing with each. The trouble with study 
or research devoted to a thing in the lump is that there 
is usually a large amount of opinion or generalization 
about the proportions of the various elements which go 
to make it up. The chemist has found it necessary to 
use the element rather than the compound as the basis 
for his science in order to be strictly objective and 
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scientific, and likewise, the scientist who deals with 
human activities must analyze them down to their 
objective elements. 

2. Greater accuracy in job description. The effort 
to tell what a school principal’s job actually is requires 
a job analysis of the principal’s work. In other words, 
it demands a careful study of the content of his job. 
This would involve making a complete list of the 
duties or functions which he performs, or the partic- 
ular activities which occupy his time. This amounts 
to a dissection of the principalship and a description 
of it in terms of specific tasks, such as interviewing new 
teachers, conferring with parents, signing attendance 
cards, etc. Until such a job analysis is made, no one 
knows exactly what a principal really does, or how 
his effectiveness can be improved. The few such 
studies that have been made have revealed that some 
high-salaried school officials, and likewise business 
executives, were doing the work properly belonging 
to low-salaried clerks and secretaries. Description of 
a job is not an end in itself, but it is extremely essen- 
tial as a basis for further scientific study of it, just 
as description of a new disease germ is basic for the 
work of the medical scientist in developing a scientific 
method of controlling it—or, to change the figure, 
dissection of jobs to discover their elements serves 
somewhat the same function in personnel science as 
dissection of organic tissues has served in the science 
of physiology. 

3. Improvement of practical efficiency. As the name 
implies, job analysis concerns the practical worker. 
It originated in the vocational and industrial world. 
It has to a large extent been used as a servant of 
vocational education, looking to improvement of the 
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performance of practical processes. It is the outstand- 
ing technique for assembling helpful material on 
methods of performing the work of a given occupa- 
tion, for which purpose it has been devised. Yet, its 
use need not be limited strictly to occupational activi- 
ties. It is applicable to the whole range of human 
affairs, to anything that involves behavior or perfarm- 
ance. Is not being a good citizen or being a lover 
of literature an activity or performance which is sub- 
ject to improvement? If so, is there not need for 
analyzing the activities of citizens or book lovers to 
find how the better practitioners of these arts perform 
them? If education is to change or improve human 
behavior, is not all education in a sense a practical 
training for increased efficiency in performing some 
kind of activity or job? In this sense, job analysis is 
basic to all education, and is, as has already been rec- 
ognized, the foundation upon which curriculum mak- 
ing must ultimately rest. 


II. Job Analysis in Personnel Work and 


Administration 


Job analysis is of fundamental importance to the 
executive or administrator. It has been one of the 
central features of various systems of scientific man- 
agement, and was used extensively by the United 
States Army in perfecting its tremendous organiza- 
tion. There are numbers of urgent educational prob- 
lems of an administrative type which call for the use 
of this procedure, of which a few are discussed in the 
succeeding paragraphs of this section. 

1. Selection of employees and vocational guidance. 
In securing a new teacher for a school, or a new mem- 
ber for its administrative staff, it is as important to 
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know the characteristics of the position to be filled 
as of the applicant who is to fill it. Matching em- 
ployees and jobs effectively calls for detailed study of 
each. The study of the applicant does not concern 
us here, since that task has been analyzed in some 
detail in the chapter on case study technique. But it 
is important to stress here the need of job analysis as 
a basis for job specifications. It is impossible, for 
example, to know whether a certain position requires 
a man or a woman, until the nature of the position 
is analyzed and clearly defined. Likewise, standards 
regarding age, training, experience, intelligence, etc., 
can be set up only in the light of the actual duties and 
responsibilities which are involved in the job. Some 
positions require high intelligence and much initiative, 
others of a more routine nature are better filled by 
persons of lesser talent who will be content to remain 
in them on low pay and without rebelling at the 
monotony involved. Which type of person to select 
for a given job can be answered only by finding out 
definitely the activities included in it. 

The other side of this same question is the problem 
of vocational guidance. In order to give intelligent 
advice to a student regarding the work for which he 
is to train, it is necessary to have definite information 
about the jobs between which he is to decide. Before 
recommending law, medicine, or teaching to a student 
in college, the duties and demands of these callings 
must be known. If a given individual has a physical 
handicap, or a speech defect, a job analysis of a 
number of callings, or a number of positions within 
a single organization, would probably reveal the job 
or jobs which such a person could fill successfully. 

By way of summary, it is reasonable to think that 
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there is a best man for every job and a best job for 
every man, and that the task of matching men and 
jobs will be greatly simplified by making a careful 
study of the exact nature of each. 

2. Conservation of talent. Much has been said in 
recent years about conservation of natural resources, 
but the conservation of human talent is of greater 
importance even than this. It sometimes occurs that 
a small man gets into a big job, but he usually fails 
to perform his duties satisfactorily and is discharged. 
But a big man not infrequently gets into a small job 
and stays in it indefinitely because no one recognizes 
his capacity for a more responsible position. Business 
concerns have been known to reduce their overhead 
charges enormously by hiring clerks to do work for- 
merly done by high-salaried executives who had really 
been doing routine work. It took job analyses by 
so-called efficiency experts to direct attention to the 
waste of talent that was going on and to bring about 
a remedy. . 

3. Determining systems of promotion. A common 
school policy in the past has been to fill administrative 
positions by promoting successful classroom teachers, 
often as a reward for work well done, yet we may 
at least raise the question as to whether there is not 
some better source for administrators. Is the teacher’s 
work the most natural and effective training for the 
work of administration? Is the principalship the nat- 
ural culmination of a career of successful teaching? 
Or is the superintendency the natural step after the 
principalship? Or should the successful elementary 
teacher be promoted to junior high school, and thence 
to high school, and finally college teaching? No 
adequate sequence of jobs or routes for promotion can 
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be determined and scientifically evaluated except in 
terms of objective studies of the duties and responsi- 
bilities of the jobs concerned. 

4, Setting limits of responsibility. One of the cen- 
tral features of scientific management is ‘‘functional- 
ization,” or establishing definite limits of responsibility 
for each workman and each executive or staff officer. 
In other words, it is a scientific rather than a hap- 
hazard division of labor between the various employees 
in the organization. Some of the published school 
surveys have revealed undue overlapping of jobs— 
so great, in some cases, that three different persons 
were responsible for the same taks. Other duties were 
discovered which were not assigned to anyone in 
particular, and were being almost entirely neglected. 
Job analysis is the method by which the executive may 
determine how to organize his staff so as to establish 
clearcut lines of responsibility, and at the same time 
have the division of labor conform to the most natural 
or efficient standards. The various duties and respon- 
sibilities cannot be parcelled out among the different 
employees until the whole list of duties and responsi- 
bilities has been determined. A job analysis of the 
“reservoir” type, which lists all the duties of all the 
workers, may be made the basis for a redistribution 
of the various jobs among the various workers in the 
light of a broad perspective or bird’s-eye view of the 
system as a whole, resulting in a more effective and 
economical division of labor and a greater degree of 
coordination of the efforts of all in the common under- 
taking. If a science of school administration ever 
comes into existence, it seems reasonable to expect 
that it will rest quite heavily upon job analysis as 
one of its fundamental techniques. 


JOB ANALYSIS 143 


III. Job Analysis and Training 


One of the principal uses of the job analysis tech- 
nique is involved in the field of methods. For example, 
if it were our desire to organize a training course for 
teachers of gifted children, or for coaches of athletics, 
or for elementary school principals, we should either 
develop the course by haphazard means or else organize 
it from the results of a careful job analysis study. 
The principal steps in the latter process are outlined 
in the following paragraphs of this section. 

1. Difficulty analysis. The first step in preparing 
a training course for any job is to discover the 
parts of the job which present difficulty, or 
which are imperfectly done. There is no need of 
giving training in matters that are already perfectly 
done. Economy demands that the major effort be 
spent in remedying the deficiencies that exist. These 
difficulties, deficiencies, or errors may be discovered 
in many ways, but making a complete list of them 
calls for a rather systematic search. The investigator 
might begin by listing all difficulties he has observed 
or experienced. He may supplement this by reading, 
by interviewing various people who are engaged in 
the job, by making systematic observations of them 
as they work, and by having them list their difficulties 
in writing. Every difficulty, however large or small, 
general or specific, important or trivial, might well 
go into this preliminary list, in order to get as many 
leads or clues as possible. It will ordinarily be impos- 
sible to follow up each of these by a search for methods 
of doing it, because of the enormous amount of labor 
involved. Nor would it be desirable to do so, because 
such a list would probably still be far from complete 
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and would thus cause a neglect of all difficulties not 
definitely included. 

This fact suggests the need of classification of the 
list of difficulties into working groups or types. When 
one hundred or more specific difficulties have been 
assembled, they will usually group themselves into a 
dozen or more major classes, after which several more 
specific difficulties will begin to be noticed to add to 
each group. For example, locating a few children 
in a family may help to locate their parents, and by 
observing the movements of the parents for a period, 
the remaining children of the family are likely to be 
discovered. This classification of specific difficulties 
into working units amounts to a determination of the 
chapter titles for the textbook which is to be prepared 
for the training course, or the topics which are to be 
made the main lesson units in the course. 

Care should be taken to arrive at the above-men- 
tioned chapter divisions by classifying specific diffi- 
culties rather than by subdividing the field as a whole, 
since the latter is almost certain to-be a structural 
rather than a functional classification, and since the 
former is much more likely to result in a group of 
difficulty-units which are mutually exclusive and all- 
inclusive. 

2. Collecting methods of solving each difficulty. 
After the difficulty analysis has been satisfactorily 
completed, the search for methods begins. In order 
to be sure that the methods discovered are truly 
applicable to the difficulties, the latter should be 
expressed in the form of questions of the “How to 
——” type, as “How to handle disciplinary cases sent 
to the office by the classroom teacher when the teacher 
is in the wrong.” Unless a difficulty can be expressed 
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in this form it is really not a difficulty, or else the 
difficulty has not yet been clearly defined in the inves- 
tigator’s own mind. Having been so defined and 
formulated, the difficulties might well be given serial 
code numbers for ready identification, and for ease in 
keeping methods associated with their respective 
difficulties. Then every method which is discovered 
may be written on a small card and given the number 
of the difficulty to which it applies, all methods for a 
given difficulty being filed in the same place. This 
process of collecting methods would be carried on until 
no new methods were to be found, or else terminated 
at such a point of thoroughness as practical consider- 
ations might require. The sources of methods would 
naturally be chiefly in the practical workers them- 
selves, as they are the people who supposedly know 
how to do the job which is being studied. Certain 
other sources may at times be available and desirable. 
The detailed technique for collecting methods is not 
given here, the reader being referred to later para- 
graphs of this chapter and also the chapters on inter- 
view technique, observation technique, and curriculum- 
making technique, for fuller directions on these points. 
The essential point here is that all possible methods or 
suggestions on how to solve the respective difficulties 
should be assembled. The basis of selection or rejec- 
tion should be relevancy, since it is important to have 
all methods at hand, however good or bad, when the 
stage of evaluation is reached. The merits of the 
different methods should not be of any great concern 
to the investigator during the collecting stage of the 
study. Evaluation should take place later, and on 
a more scientific plan than is feasible during the 
collection stage. 
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3. Evaluating alternative methods. The usual re- 
action of a person who has conducted a job analysis 
study involving the collection of methods of solving 
a set of difficulties is something like this: ‘I didn’t 
realize there were so many different ways of doing 
the same thing.” .How, then, are we to know which 
is the best of several ways of doing a thing? There 
are several possible ways of evaluating them: (1) 
They may be compared experimentally or statistically, 
under carefully controlled conditions, with strict forms 
of measurement, and their relative merits established 
by objective and thoroughly scientific evidence. This 
is usually a laborious task, and no one investigator 
would be able to conduct more than a very few such 
experiments, even if he devoted his whole life to the 
task. This must ultimately be done, however, in order 
to get final and absolutely dependable evaluations. 
(2) The methods reported by highly successful workers 
may be contrasted with those of workers of lesser 
ability. Comparisons of extremes in this way may 
shed considerable light upon the relative values of 
the methods they employ, although it does not neces- 
sarily establish the facts conclusively. (3) The reasons 
for the various methods may be collected along with 
the methods themselves, and thus form the basis for 
an empirical or common sense sort of evaluation, 
which, while not final and scientific, is better than no 
guidance at all. (4) The various methods may be 
tested in the light of the fundamental psychological 
and philosophical principles which are commonly 
accepted as true, the investigator doing most of the 
work of judging. (5) The entire list of methods may 
be submitted to a group of successful workers in the 
field for rating or ranking as to their relative merits. 


JOB ANALYSIS 147 


It will frequently happen that one investigator’s 
time will be sufficient only to complete the collection 
phase of the study, as in the case of a student working 
on a master’s thesis problem, and that the other steps 
will have to be completed by other investigators. In 
such cases, care should be taken to present the methods 
collected simply as alternative or proposed methods, 
and to avoid leaving an impression of finality about 
them. Until methods are evaluated, they should be 
clearly branded as tentative and might well be labeled, 
“To be used at your own risk.” It is better to have 
available a number of mutually contradictory and 
untried methods as a source of suggestions than to 
have nothing at all upon which to rely for guidance 
in facing a difficulty. 


IV. General Problems of Procedure 


Having treated briefly a few of the special consider- 
ations involved in job analysis in administration and 
in training, we are now ready to treat some of the 
general problems and procedures in actually making 
a job analysis investigation. Many of these procedures 
will be as applicable to a study for one purpose as for 
another. 

1. Limiting the field of investigation. There is 
nothing to be gained by making a job analysis for 
its own sake. Unless there is a definite purpose behind 
it, it should not be undertaken. If there is such a 
purpose, the investigation should be limited to a scope 
that conforms to that special purpose. For example, 
an industrial concern may be primarily concerned about 
wages, or accidents, or labor turnover. Or an educa- 
tional institution may be concerned with reducing 
pupil failures, saving building space, or reducing the 
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cost of instruction. For practical purposes, the indi- 
vidual investigator, working alone, should limit his 
efforts to a single problem of a rather narrow scope 
and do that one thing well. Also, as a rule, the study 
should be limited to a functional analysis rather than 
attempt a complete structural analysis. That is, the 
point of view should be that of behavior units rather 
than that of logical divisions of the field. 

2. Getting access to the needed information. There 
are many ways of getting the actual information 
needed in making a job analysis. A few may be 
briefly mentioned here. Several of these are discussed 
at greater length in other chapters. 

(1) Personal experience. Efficiency engineers and job 
analysts are not uncommonly persons of considerable 
experience in the general type of work they are inves- 
tigating. If an analysis of methods of teaching 
Spanish is to be made, it is ordinarily best for an 
experienced Spanish teacher to conduct the analysis. 
In the absence of such experience and direct contact, 
it has at times been found worth while to go into the 
job as an apprentice and make the job analysis con- 
current with the learning of the job. In the industrial 
field it has been necessary at times for efficiency 
engineers to work as common laborers in order to 
get the intimate acquaintance with the work that was 
needed. 

(2) Observation. Next to doing’ the job himself, 
comes observation by the investigator of other persons 
as they do the job. It reveals some things that are 
not discovered by practical experience on the job, 
because a more detached and impersonal viewpoint is 
possible. It also yields richer results because it makes 
available the experience of many workers instead of 
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only one. When it is supplemented by the use of stop 
watches, photography, etc., it may be of enormous 
value. 

(83) Workman cooperation. When the experience and 
observation of the investigator are supplemented by 
that of the various individual workers, the result is 
very much enriched. The workers should ordinarily 
be taken into the investigation, their whole-hearted 
consent gained, and enlisted in an active effort to 
secure complete information on the problems being 
attacked. Anything that arouses suspicion or which 
resembles an attempt at exploitation is naturally 
resented and causes the doors to be barred against 
access to the desired information, and there is much 
needed information that can be gained from no other 
source than the actual workman on the job. 

(4) Personal interviews. Many facts about a job 
which would naturally escape the eye of an observer 
will be discovered through personal interviews. The 
presence of the interviewer makes possible a thorough 
searching of the experience of the person interviewed, 
until the desired information is brought to light. It 
is a device for securing the results of self-analysis 
of the persons interviewed and also of their observa- 
tions of their fellow-workers. 

(5) Written reports or questionnaires. The use of ques- 
tionnaires in job analysis, as elsewhere, is subject to 
question, and should be resorted to only after due 
consideration. Its advantage of economy of labor is 
usually more than counterbalanced by its disadvantage 
of reduced returns. Only when there is some form 
of exceptionally strong authority or motivation, can 
it be made to yield worth-while results. 

(6) The use of assistants. Some of the best and most 
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extensive job analyses have involved the services of 
numbers of hired assistants to make the necessary 
observations, interviews, etc. This device makes pos- 
sible a much more comprehensive and thorough analy- 
sis than would otherwise be possible, and if the assist- 
ants are carefully trained so that their methods are 
uniform and on a high plane, their work need not be 
inferior in quality to that which the investigator would 
do himself. There are possibly many cases in which 
a group of students working under the direction of a 
well-trained teacher could carry out a job analysis with 
profit to themselves and perform a real service to 
science. 

(7) Tests. Sometimes the information needed can 
be gained in a better way by actual tests than by ob- 
servations and interviews. For example, if we desired 
to learn the amount of mathematical ability required 
of a bookkeeper, it would probably be best to adminis- 
ter well-chosen mathematics tests instead of asking 
him how much mathematics he thought he needed or 
used. Special tests would have to be made for some 
problems, but the trouble would be justified by the 
improved results. 

(8) Reading. Something in the way of published in- 
formation is available for practically every job imagin- 
able. In such fields as teaching and administration, 
a rather extensive occupational literature has been 
built up. Two cautions should be observed regarding 
the use of this material in job analyses. One is the 
caution against ignoring or slighting it. The other is 
the caution against using it to the total exclusion of 
other sources of information. As a general rule the 
best and most helpful information about how to do 
any job is to be found in the unrecorded experience of 
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those who are doing the job, rather than in the pub- 
lished literature on that occupation. 

(9) Original thinking. The job analysis investigator 
must be more than a collector and compiler of meth- 
ods. He must, to a certain extent, become an inventor 
of methods, using the suggestions of others as raw 
material to combine in new ways. By the time he has 
worked intensively on a problem or job for a long 
period of time he is probably more fertile with sugges- 
tions for new procedures than any other person, and 
it would be unfortunate if, in his effort to be objective 
and impersonal, he should fail to record the results of 
his own thinking about the job. The fact that methods 
which he invents are not actually practiced by any 
single workers at present need not be a serious objec- 
tion, since the evaluation of methods is a different and 
later process than the collection of methods. 

3. Securing variety of samplings. Representative- 
ness is as essential in job analysis as in other forms of 
research. For example, it would be unwise to base a 
job analysis of college training upon a single collegiate 
institution and then draw inferences from it regarding 
college teaching in general. Furthermore, in order to 
get the richest and most complete information about 
a job, it should be analyzed as carried out under widely 
varying circumstances. A course on salesmanship 
would not be as rich in helpful procedures if it were 
derived solely from salesmen behind the glove counter 
as if it had been based upon the work of salesmen of 
suits, furniture, real estate, stocks and bonds, and so 
on. Methods secured from beginners will supplement 
the returns from persons of long experience, because 
they are keenly conscious of certain things they do 
which experienced persons have already automatized 
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and forgotten. Elementary school teachers will con- 
tribute methods not given by high school teachers, and 
principals can give points on teaching as seen from the 
administrator’s viewpoint which would not be gained 
from the teachers themselves. Customers can give in- 
formation on some points of salesmanship which sales- 
men themselves would not think of, just as students 
can about methods of teaching. The total composite 
of activities, difficulties, or methods is greatly increased 
when the variety of types included in the sampling of 
workers in a job that is analyzed is increased. 

4. Resorting to judgment and inference. There is 
a strong temptation to list imaginary requirements or 
elements of a job as real ones. For example, it seems 
reasonable that an engineer would need to use a great 
deal of mathematics, and unless the investigator is 
especially on his guard, he may rate mathematical 
activities higher in the engineer’s job than the actual 
facts warrant. Then, again, there is a strong tempta- 
tion to make a wild mental leap from what a man 
does to what he ought to study to prepare for his job. 
Against such errors on one extreme are errors of exces- 
sive objectivity on the other extreme. A complete job 
analysis must take account of many of the personal or 
human elements in addition to the purely mechanical 
and strictly objective ones. The things a worker feels, 
wants, likes, and aspires to are as essential to an 
undertaking of him as are the things he does with his 
hands or the devices he employs in manipulating his 
tools. Frequently the only way to discover the former 
is by the use of judgments and inferences based upon 
the latter. Many of the most important elements in 
a job are of this subjective sort. The investigator 
will, therefore, be obliged to proceed thoughtfully and 
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critically and be alert to discover what is only sug- 
gested or implied as well as what is obviously present, 
but always on guard against unrestrained flights of 
imagination, or deception by misleading suggestions. 

5. Composite versus typical lists of elements. Job 
analyses fall into two general classes as regards their 
purposes. One is the class which seeks to get a com- 
posite list of all the elements in the job as it is prac- 
ticed by a large number of workers, or a list of all the 
methods known by a number of workers, regardless 
of the frequency of occurence of a given element or 
method. In other words, this type seeks to discover 
the complete range of elements, or to be all-inclusive. 
The other type of job analysis seeks to list the things 
that are found to be common to all or to a large portion 
of the different workers, and ignores those elements 
or methods which are peculiar to individuals or small 
groups. Whether it is better to make a composite list 
or a typical list depends on the purposes of the par- 
ticular study, but usually the composite list is an essen- 
tial preliminary to the typical list. That is, it is gen- 
erally best to discover all the elements or methods first 
and then conduct a special investigation of a more 
statistical sort to discover the frequency of occurrence 
of the various individual elements in the lives of the 
different workers. This point will be elaborated a bit 
further in the paragraph on the use of checking lists. 

6. Keeping records. The records for a job analysis 
are ordinarily more bulky and voluminous than for 
most other types of research. The problem of pre- 
serving order and system is accordingly an unusually 
important one. The following points suggest a few 
things that experience has shown to be of value in 
this connection. (1) Reduce the results to writing. 
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It is absolutely hopeless to trust the memory because 
the mass of detail is entirely too great. (2) Adopt 
some kind of code numbers or other observations to 
stand for much-used items in order to reduce writing 
as much as possible. (8) Keep all notes numbered or 
identified objectively so that they can be sorted and 
classified for filing without reading their contents. 
(4) Put the record for each element, difficulty, or 
method on a separate small card to facilitate ready re- 
combination or reclassification, and to facilitate the dis- 
carding of duplicates. (5) Avoid throwing two or 
more closely related points or elements together into 
one entry in the records. If a separate entry is made 
for each they can still be combined later if necessary, 
but if once combined, they can never be separated if 
later needs call for it. This idea has been expressed 
by the old saying that “you can’t unscramble eggs’. 
(6) Be sure the methods or practices are described so 
completely in the records that the description would 
enable other persons to repeat the methods or prac- 
tices in exactly the same way. Otherwise, there is 
no certainty that the methods or procedures which are 
recommended for general adoption will be correctly 
understood or adopted by those who are to use the 
results of the study. (7) Either keep a record of the 
new contributions received from each additional worker 
analyzed in order to know when the point of complete 
saturation is-reach, or else continue the analysis far 
enough to be sure that the saturation point has been 
reached or passed. As a general but only approximate 
rule, thirty persons of a given class or experience back- 
ground will contribute enough elements so that a com- 
posite list will not be materially increased by includ- 
ing other persons of that same class or group. 
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7. The use of the check list. As was mentioned pre- 
viously, it is sometimes desirable to know not only the 
complete list of all activities or methods that are rep- 
resented by a group, but also their relative frequency, 
amount, or importance. For this purpose the check list 
is of considerable service. It may be briefly described 
as a list of all items on the composite list arranged in 
a form convenient for checking by the individual 
worker beside each item. For example, the list might 
contain twenty-five tool operations of a brick layer 
for each individual bricklayer to check or rank 
as to their status in his own individual work. Such a 
list must of necessity be compiled by job analy- 
sis procedures in the first place, but when it is 
once prepared, it makes possible a quantitative in- 
vestigation and evaluation of the material included in 
it which is often of very great service. F.C. Ayer has 
made such a study of school administration, in which 
he assembled a list of one thousand specific adminis- 
trative tasks that have been known to be performed by 
one or more superintendents or principals. He then 
printed these in pamphlet form and had large numbers 
of administrators check the approximate number of 
times they did each task per day, week, or year. The 
amateur in job analysis would possibly attempt to 
combine these two phases of the study into one as he 
went along, but doing so would only hamper progress 
in preparing the original composite list without yield- 
ing any quantitative data that would have scientific 
value. The use of the check list is a comparatively 
simple task, once the composite list has been made. 

In this connection a warning should be given against 
attempting to use the check list as a means of prepar- 
ing the composite list in the first place. If a list of 
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one hundred items is presented to a man, and he is 
asked to check those which apply to his job and to add 
to the list other items not already included, he may 
do the checking satisfactorily, but he is almost certain 
not to add much to the list. His new contributions are 
easier to get if he is not confronted with a previous list. 


EXERCISES AND QUESTIONS 


1. How does the need for job analysis in education compare 
with the need for it in the industrial world? 


2. What is an “efficiency expert,” and how does he work? 


3. What are the peculiar dangers that lie in the job analysis 
type of research as regards its strictly scientific charac- 
ter? How can these be avoided? 


4. How does the need for job analysis in administration com- 
pare for the need of it in training? 


5. Referring to the chapter on curriculum-making technique, 
see how many parallels you can draw between it and job 
analysis technique. 


6. Read several authors on job analysis and compare the dif- 
ferent meanings which they associate with the term. 
What other terms do you find used to express the job 
analysis idea? Which of these seems preferable? 


7. Make a list of research problems presented by your school 
situation that might be attacked by the job analysis 
technique. 


8. What would you consider the best way to prepare for the 
making of a job analysis investigation? 
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INTERVIEW TECHNIQUE 


1. Selecting subjects to be interviewed. The policy 
to be employed in selecting persons to be interviewed 
depends upon the purpose of the interviews. The fol- 
lowing points, however, will be of some importance in 
most cases: (1) The persons interviewed should know 
a great deal about the matter upon which the interview 
is to be based. They should have had experiences 
which are worth sharing, and which are really import- 
ant. They should be more or less expert in the things 
about which they are to report. (2) They should rep- 
resent a variety of types, or a wide range of experi- 
ence. The aim in most interviewing is to get diversity 
rather than uniformity, and hence the more varied are 
the viewpoints and backgrounds of the subjects inter- 
viewed the richer the returns. For this reason it is 
often well not only to interview teachers but also super- 
visors, and even pupils, in order to get the most com- 
plete returns upon a question about methods of teach- 
ing. (8) It is usually better to hunt for “big game”. 
The persons of larger calibre have more to contribute, 
and, contrary to what might be supposed, are more 
likely to be courteous, considerate, and willing to help. 
They have become accustomed to being called upon for 
conferences and have a portion of their time definitely 
set apart for such things. (4) One interview may be 
used as a means of discovering “leads” to other prom- 
ising sources of information. Remarks made in the 
interview may suggest other persons who would be 
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likely to have valuable information. In fact, it may 
be well at the close of each interview to ask for such 
suggestions and write them down. Care should be 
taken, however, not to let the giving of such informa- 
tion be used by the subject as a means of dodging the 
task of giving what he himself has to offer. 

2. Enlisting cooperation. The success of the inter- 
view depends on how well the interviewer ‘sells him- 
self” to his subject. It is often necessary to give con- 
siderable thought to the methods to be used in break- 
ing down mental resistance and getting free and un- 
restrained responses. The following are some sug- 
gested ways of doing this: (1) The interviewer should 
think the matter through for himself quite thoroughly 
and know enough about the questions to be able to 
satisfy the subjects to be interviewed that he is cap- 
able of using the replies intelligently. (2) The inter- 
viewer should be keenly interested in the questions 
himself, else he cannot expect his subjects to be inter- 
ested. (38) The interviewer should take his subjects 
into his confidence, telling them the nature of the 
problem being investigated. Any trace of secrecy im- 
mediately sets up psychological barriers that are almost 
insuperable. (4) It is usually better to introduce a 
considerable element of formality. That is, the inter- 
viewer should make it plain that he wishes a serious 
interview rather than a casual conversation. Such 
plans as meeting at lunch tables for an “interview” 
while eating a hurried meal are unfavorable to setting 
up the kind of mental attitude desired. 

3. Getting introductions. The securing of an inter- 
view is sometimes aided if the interviewer is introduced 
to his subject by some third person who is in a position 
to sponsor him or his investigation. The faculty mem- 
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ber who is directing and supervising the research is 
usually in such a favorable position, and can do much 
to smooth the path for the interviewer. The greater 
the prestige of the persons arranging the introductions, 
or the closer their acquaintance with the subjects to 
be interviewed, the more effective are the introduc- 
tions. Not uncommonly the interviewer will find it 
best to go directly to a superior official, like a school 
principal, present his case, enlist his cooperation, and 
get him to arrange with those working under him for 
the desired interviews. Again a form of implied intro- 
duction that is almost always possible is that of telling 
the subject who is to be asked for an interview the 
names of persons who have already granted interviews. 
This serves somewhat as a testimonial to the worth- 
while nature of the project, and is a fairly good opening 
wedge. 

4. Appealing to personal pride. Vanity seems to be 
an inherent attribute of practically all people. There 
are numbers of ways in which the interviewer can use 
it to further his ends. For example, he may find out 
beforehand what special thing his prospect has done, 
or what special kind of information he possesses, and 
approach him from that angle. The prospect is natur- 
ally pleased to know that his expertness in this line 
has been recognized and that he is considered an 
authority on the subject, and in his general feeling of 
expansiveness is more likely to grant an interview. 
This reaction is enhanced if the interviewer makes it 
clear that not just anyone and everyone is being asked 
for interviews, and that the requests are being made 
of only a selected few who have much to contribute. 
The same effect can be maintained during the interview 
itself by the simple device of reacting to the subject’s 
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statements as if they were weighty contributions and 
recording voluminous notes upon them whether they 
are of any importance or not. 

5. Making appointments. A definite appointment for 
an interview should usually be made _ beforehand. 
Chance meetings are seldom satisfactory. A personal 
call to ask for an appointment is almost always pre- 
ferable to a request by telephone or letter. It is well 
to ask for more time than will probably be needed 
rather than less, because this emphasizes the import- 
ance of the project and produces a more favorable 
attitude. The interviewer should be willing to make 
any reasonable arrangement for a meeting that is sug- 
gested by the prospect, considering his own time and 
convenience of no importance. To get impatient or 
cross because a person forgets his appointment or is 
delayed is distinctly out of place, since “beggars should 
not me choosers.” If a person is unable to grant an 
interview at the time asked for, it is best to set a 
definite date as far ahead as necessary, and then to 
call him on the telephone the day before to remind him 
of it. It is especially bad policy to agree to meet “‘some- 
time” or to depart with the understanding that the 
prospect will call up the interviewer when he has a 
convenient time for the interview. Such an arrange- 
ment takes the situation out of the control of the inter- 
viewer almost entirely. The time set for an interview 
should be such that there will not be a rush or hurry 
to get through. It is better to postpone the interview 
entirely than to sacrifice the opportunity to get its full 
value by attempting it under unfavorable conditions. 
The same considerations apply also to the selection of 
a place where the interview is to be held. At the close 
of an interview, it is well to leave the way open for a 
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second interview in case it is discovered later that any 
essential items have been overlooked. 

6. Framing questions. The interviewer should have 
definite questions in mind to ask, instead of taking each 
subject’s ideas according to the order or arrangement 
of the giver. The questioning should be planned with 
great care to draw out as much and as valuable infor- 
mation as possible. Even in cases where reasons of 
tact or diplomacy require that questions be kept in 
the background and that the interview progress as a 
“spontaneous” conversation, the questions should be 
distinctly present in the mind of the interviewer. In 
some cases the questions should be made in full and 
then summarized each into a key word or phrase which 
is meaningless to anyone except the interviewer in 
order to provide definite guidance without playing into 
the hands of the person interviewed. A good rule in 
framing the questions is to confer with someone or 
try them out in a rehearsal interview and revise before 
going into the regular series of interviews. 

7. Number of questions. The matter of number of 
questions has two aspects. (1) How many questions to 
include in the entire list. (2) How many to ask any 
single individual. The number to include in the entire 
list depends upon the degree of thoroughness with 
which the study is to be made. The finer the analysis 
of the total problem, the greater wealth of detail will 
be secured, but the greater the labor of getting answers 
to all the questions. The more questions there are in 
the list, the more nearly alike are the individual items. 
In fact, it is possible to split questions up into parts 
that are so small that the ordinary person who answers 
them will be unable to see how they differ from one 
another, and will tend to make the same response to 
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several different kindred questions. The writer’s ex- 
perience has been that it is usually best to have one 
question for each chapter in the book or thesis which 
is to be based on the replies. This means that the 
field will ordinarily be divided into not more than fif- 
teen or twenty divisions, with one question to each 
division. The other phase of this problem is how many 
questions to ask in any single interview. The answer 
is, from seven to twelve, as a general rule. To attempt 
more than this results in fatigue and reduced quality 
of answers to the last questions. It is better to ask 
only half the questions in the entire list and come 
back for the other half, or to ask another person the 
other half, than to rush through a long list super- 
ficially. The longer the interview dwells on a single 
question, the richer the returns that are secured. 


8. Number of interviews. The fruits of successive 
interviews are subject to the law of diminishing re- 
turns. The first one yields all new material. The sec- 
ond adds some new material and some that duplicates 
the first, and soon. Charters has suggested that about 
thirty interviews on a given question is a suitable limit, 
and that further interviews after that point are often 
not worth the time required. Mrs. Woodruff* kept a 
record of the number of new ideas gleaned from each 
of her interviews, and found that while the 27th and 
28th interviews yielded 24 and 29 ideas respectively, 
they yielded no new ideas at all. If further information 
is desired after reaching such a point, it is obvious that 
additional interviews of this type cannot be expected 
to supply it. It is necessary to adopt a different 


*Hazel M. Woodruff, Method of Teaching Current Events. 
Unpublished master’s thesis, University of Southern California, 
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method, a different set of questions, or else go to per- 
sons of a different background of experience who will 
not simply duplicate the answers of the previous ones. 
If a large number of persons are to be interviewed, it 
is better to divide up the questions and the subjects 
into sections, and interview each section more inten- 
sively on its special portion of the entire list of ques- 
tions. 

9. Self-interviewing. Before the interviewer pro- 
ceeds to ask other people his questions, he should inter- 
view himself very thoroughly. If he writes out his own 
answers he has a better understanding of the issues, 
as well as a greater capacity to appreciate the mental 
states of those who are to respond. Furthermore, if 
his own ideas are to be preserved at all, they must be 
recorded before the ideas of other persons are secured, 
else the two coincide and originality is sacrificed. Self- 
interviewing should not cease, however, when the inter- 
viewing of others begins. The interviewer should 
record his own ideas whenever they occur to him, 
whether in the midst of an interview, while writing up 
the results later, while organizing the returns from sev- 
eral interviews, while walking through the park, or at 
any other time. By the time he has given a problem 
as much consideration as is involved in a series of 
interviews, he naturally has some valuable and original 
ideas upon it which it would be unfortunate indeed to 
discard. 

10. “Digging deeper”. The first reply to a question 
is likely to be the natural result of the laws of associa- 
tion, and will bear a striking similarity for the various 
persons interviewed. In order to get more than this 
superficial or conventional reply, it is necessary to dig 
deeper into the subject’s experience by follow-up ques- 
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tions until those unique contributions which are pecu- 
liar to him have been discovered. The ability of the 
interviewer is quite effectively measured by his ability 
to hang on to’a question until it yields valuable in- 
formation. If a reply is not understood, or if it seems 
to lack significance, further questioning is required to 
bring out the real idea behind it. If a person’s mind 
is kept upon a given question long and persistently 
enough by skilled interview technique, it is almost cer- 
tain to yield something that will be of value to the 
interviewer. 

11. Varying the terms. When a given question has 
elicited all the information of which it seems capable, 
it is time to state it again in different words and thus 
get a second installment of information. The same 
question may be presented over and over in this way, 
each time bringing forth new material. Ideas that are 
not aroused by one word may be aroused by its syn- 
onym. Thus, to ask for “methods” gets one response, 
“devices” another, and “‘suggestions” still a third, and 
a question has not been exhausted until it has been 
asked in several such ways. This idea also has an im- 
portant corollary in the use made of the replies re- 
ceived. The same general thought may be contributed 
by several different persons, each in slightly different 
phraseology. To record it only. once, and ignore it 
thereafter as a duplication, is very unwise. It is better 
to record it each time, in the phraseology of the 
speaker, and then to assemble all these various kindred 
statements later, where the slight differences will call 
up in the mind of the investigator their slightly differ- 
ent shades of thought. 

12. Asking how. A common error of interviewing 
is that of asking ‘Do you ” rather than “How do 
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you The former type yields simply a yes or no 
answer, while the latter yields valuable methods for 
the guidance of others. If the purpose is to make a 
survey of prevailing practice, calling for the “Do 
you ” type of questions, the interview method is 
not suitable, because it is too laborious to permit the 
recording of enough cases to afford an adequate samp- 
ling. Such questions had better be presented by ques- 
tionnaires and answered by several hundred persons. 
Not only should the interview question be stated to 
include the word how but the interviewer should make 
extra effort to insist that the answers received actually 
answer that question. Also, it is frequently necessary 
to ask additional how questions with respect to a 
method that is reported, in order to get more detailed 
information on how to utilize that method. 

13. Precept versus practice. Whether to ask ques- 
tions in the form ‘How to 2 0re HOW -d0;y0u-———* 
involves a very fundamental issue in this type of re- 
search. The issue is whether to find what is actually 
practiced, or to find what are the possible or desirable 
ways which might be practiced. If the goal is the col- 
lection of facts, it is better to ask a person how he does 
the task; but if the goal is to collect suggestions, it is 
better to ask him what are the various ways of doing 
it. The latter obviously yields a wider range of meth- 
ods, while the former yields selected methods that have 
to some extent been tried out and found to be workable. 
There are cases, however, when asking only for how 
a person does a task would uniformly neglect the best 
methods, as in the case where the best methods have 
not yet been adopted in actual practice. The writer’s 
preference in most cases is for the “How to ” type 
of question. 
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14. Examples. For each method secured in an in- 
terview, it is desirable to get at least one illustration. 
The simple device of asking “For instance S408 
enormous value in improving the results of interviews. 
Without it the points are likely to be somewhat ab- 
stract at the time, and possibly meaningless weeks 
later when they are being assembled and organized into 
the final report. 

15. Asking why. If methods and policies discovered 
through interviews are to be of service in the guidance 
of other people, they must be interpreted, explained, 
psychologized, or justified. Merely to know a way of 
performing a task is useless unless there is a reason or 
motive for adopting that way. It is for this reason 
that the interviewer should not be satisfied when he 
has learned how, but should go a step further and get 
the special merits or advantages of the methods re- 
ported. The reasons for not doing it a certain way are 
as important as the reasons for doing it that way, since 
the disadvantages or weaknesses of a method may be 
as important to know as the advantages. 

16. Keeping on the subject. It is the interviewer’s 
responsibility to guide the course of the interview, 
and to bring it back if it tends to wander. This re- 
quires clear thinking and a logical turn of mind, or else 
the returns from the persons interviewed will be in a 
muddled state. There are a few devices, however, 
which may prove helpful in holding the interview upon 
the subject. To have the list of questions prominently 
displayed is a constant reminder that there is a definite 
business to be attended to, and by constantly referring 
back to this list, it ig possible to break up irrelevant 
trends, and to bring the subject back to the desired 
topic. Without such a list of questions or topics to 
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lean upon, it is at times difficult to direct the course of 
the conversation properly without seeming a bit rude or 
autocratic. 

17. Flexibility. A rigid adherence to a prepared list 
of questions is seldom wise. It is well to skip about in 
the list, if by doing so a better set of responses can be 
secured. The principle, ‘strike while the iron is hot,” 
also necessitates following up a train of thought at the 
time it is under way instead of shutting it off merely 
because it is not the next thing on the list. Someone 
has compared interviewing to boxing, in that it is 
necessary to take advantage of the openings when they 
occur, lest by a delay the opportunity be completely 
lost. It sometimes happens that a certain question is 
particularly suitable as an opener for the interview for 
one person, and another question for another, accord- 
ing to their special interests and backgrounds. As in 
the case of observation, it is sometimes best to take the 
impressions or statements from the person interviewed, 
just as they come, and to fit them into their appro- 
priate categories later at leisure. At any rate, it is 
necessary in practically every interview to take up a 
new piece of paper and record material that answers a 
later question and then when that train of thought 
runs down, come back to the question that was being 
discussed before the interview wandered. 

18. Arguing. A good slogan to keep in mind is that 
“the person interviewed is always right’. It is wise 
to question or doubt the views expressed only when 
such a doubt is used to bring out the reasons back of 
the views expressed. The interviewer should be eager 
-to learn rather than to teach, and to avoid imposing 
his views or opinion upon the person interviewed. 

19. Revealing the questions beforehand. There are 
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times when it seems best to leave a list of the interview 
questions to be studied over before the interview. This 
improves the results, in that it, at least theoretically, 
causes more thinking to be done on the answers. But 
as a matter of fact, it usually detracts more than it 
adds. Usually the questions are considered only super- 
ficially if at all before the interviewer arrives. If, how- 
ever, they are studied over with great care before the 
interview, the result is not pure gain. In such a case 
the interviewer gets the end result of the thinking pro- 
cess and misses the intermediate steps in the thinking 
which were carried out before he arrived on the scene. 
These intermediate steps are often as valuable or more 
valuable than the final decisions to which they lead. 
There is a yet more serious disadvantage in leaving 
the question beforehand. It takes the control of the 
interview out of the hands of the interviewer and puts 
it in the hands of the subject. Or, at best, it results 
in a dual control, which greatly handicaps progress. 
If only the interviewer has possession of the questions, 
he can hang on until he is satisfied, but if the subject 
has them, he can set a more lively pace and bring the 
interview to a close before the interviewer has secured 
all the information he needs. 

20. Confidential information. Before best results 
can be obtained from some people, it is necessary to 
assure them that “What they say will not be used 
against them.” Usually it is only necessary to give 
assurance that names will not be published in connec- 
tion with statements, but in other cases, it is well to 
give the person involved an opportunity to read and 
authorize, censor, or disapprove the publication of in- 
formation supplied in the interview. 

21. Open versus secret note-taking. There are times 
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when note-taking would defeat the purpose of the in- 
terview, and when it is better to remember as well as 
possible, and record later. Sometimes the interviewer 
resorts to playing with his pencil on a piece of scratch 
paper, making marks which appear to be meaningless 
but which help him to remember. In most cases, how- 
ever, it is better to secure the full cooperation of the 
subject interviewed and then to take the notes openly. 
In fact, as a general rule, the more pronounced the 
note-taking procedure, the more important does the 
subject feel and the more willing he is to open up his 
heart and express himself. 

22. Recording responses. Whenever possible, it is 
well to write down the statements in the speaker’s own 
words at the time. The value of this, as suggested by 
Charters, is to ‘“‘retain the flavor of the language and 
the exact terminology of the person interviewed.” The 
written record should be so full and complete as to 
make unnecessary any later supplementing from mem- 
ory. If only a few outstanding things are recorded 
and the rest added later from memory, there is con- 
siderable danger that the interviewer will read his own 
ideas and impressions into what he writes, and miss 
much of the real contribution. Naturally, shorthand 
is a help in this kind of work, but if complete steno- 
graphic reports are to be made, it is necessary to have 
a stenographer along so the interview’s mind may 
be free to direct the discussion. Also the complete 
reports are usually so padded with unimportant ma- 
terial that the labor of using them is enormous. 

23. Card systems. Either at the time of the inter- 
view or later it is well to record each point or idea 
upon a separate small card or slip of paper. A good 
plan is to assign each question a serial number and to 
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memorize this code thoroughly. Then each card can be 
identified by its appropriate number in the upper left 
hand corner at the time the point is written down. As 
many cards of a given number would be used as there 
were points, ideas, or contributions bearing on that 
question. Then later all cards of a given number from 
all interviews could be brought together conveniently 
for organization. If the notes are taken on larger 
papers during the interview they should be transcribed 
to cards as soon afterward as possible. 

24, Organizing replies. If the replies are reduced to 
cards as just suggested, the task of organizing is ex- 
tremely simple. It is best to wait until all replies to 
all interviews have been assembled before beginning to 
organize any of them, as it is useless to start crystal- 
lizing the material if more is to be added later. Also 
the write-up for one question will depend somewhat 
upon that for the others, and there will be problems of 
overlapping, conflicts, and establishing cross-connec- 
tions. When ready to begin the actual arrangement of 
the material, the cards can simply be sorted into little 
piles according to their types, with illustrations and 
reasons bunched with the methods to which they apply, 
and these little bunches or piles may then be used as 
guides in the writing of actual paragraphs of the re- 
port. 

25. Evaluating suggestions. The returns from inter- 
views can seldom if ever be considered as strictly sci- 
entific. Whether one person’s methods are better than 
another’s is a matter for testing by some technique. 
The task of the interviews is to collect, assemble, and 
classify the various alternative methods and then turn 
the classified list over to the experimental scientist or 
the statistician, or the psychologist, to complete the 
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task of evaluation. The interviewer should, before do- 
ing this, however, take the trouble to make such inter- 
pretations of a common-sense or scientific nature as 
he can before leaving the job. Or, better still, he may 
at this point assume the role of the experimenter and 
carry his work on through the final stages. If this is 
impossible, however, it is better to have collected and 
classified a body of empirical or common-sense methods 
than to have left them only in the unrecorded experi- 
ence of isolated practitioners. 

26. Recording the interviews. The faith which the 
reader will put in a book which has been written from 
interview materials will depend upon how thoroughly 
the interviewer has done his work. Consequently, it is 
necessary to accompany the book by a record of the 
procedure and sources of data used. In other words, a 
record should be made at the time of the interview in- 
cluding the name, initials, title, address, occupation or 
position, specialty, etc., of the person interviewed, the 
date, place, and amount of time consumed in the inter- 
view, and any similar information which sheds any 
light on the thoroughness or system with which the 
study was pursued. If it is so desired, each person may 
be assigned a code number, which, when written in the 
upper right hand corner of each card, would identify 
the source of each of his contributions. This makes 
possible giving credit for the ideas without the labor of 
writing the name on dozens of cards. The final report 
should ordinarily bear a list of the persons interviewed 
simply as a means of expressing appreciation and giv- 
ing credit for the courtesies shown. 
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EXERCISES AND QUESTIONS 


1. How does the newspaper interview differ from the research 
interview? What procedures or policies do the two have 
in common? 


2. What types of research problems require that interviews be 
conducted in disguise, and what types lend themselves 
to frank and unconcealed methods? 


3. What are the advantages and the disadvantages of inter- 
views conducted “spontaneously” and without the person 
interviewed knowing that he is being interviewed? 

4, List the prerequisites of a successful interviewer. 


5. List a number of educational research problems that are well 
suited to the use of interviews. 


6. Read several treatments of the subject of interviewing and 
classify the types of interviews discussed. 


7. How can the results of interviews be made to possess greater 
scientific value? 

8. How does interviewing compare with other methods of get- 
ting information as to economy of time and effort? Ex- 
pense? Richness of returns? Practical worth of returns? 
Scientific nature of returns? Personal development of 
the research worker? 
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QUESTIONNAIRE TECHNIQUE 


1. Limitations. According to some people, the most 
important piece of advice about questionnaire technique 
would be to avoid it entirely. It is quite true that it 
has been seriously and frequently abused, and that its 
use by untrained and uncritical workers has brought it 
into general disfavor. It should not be used as a lazy 
substitute for other methods, nor to collect facts that 
are available in print or in official records which are 
reasonably accessible. The proposed questionnaire 
should be clearly capable or producing the desired re- 
sults or of actually solving the problem before being 
undertaken, and there should be no other possible way 
of getting the desired information. Ordinarily, a care- 
ful bibilographical survey and a thorough study of the 
problem by means of secondary sources would advance 
the investigator’s knowledge to such a point that he 
would not need to send a questionnaire, but would in- 
stead narrow the question down somewhat, and solve 
it by more refined methods, such as are described in 
other chapters of this book. In the few instances in 
which this is not the case, and in which experimental, 
historical, statistical, and similar techniques are not 
possible, the following paragraphs may have some 
practical value. 

2. Securing an adequate response. If the returns are 
to justify worth-while conclusions, they must be based 
upon a sufficient number of replies to give a high degree 
of reliability. Not only should the replies be numerous 
but also complete. That is, they should represent a 
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high percentage of response to the blanks sent out. The 
lower the percentage of replies the greater the danger 
that the answers are selective and therefore not repre- 
sentative of the total group of persons requested to 
answer. It is sometimes decidedly better to sacrifice 
numbers in order to gain a high percentage of response. 
This can be done by concentrating more energy upon a 
few representative persons and getting them all to 
respond, instead of using less pressure and applying 
it to more persons. There is always grave doubt at- 
taching to the unanswered questionnaires, or to the 
unanswered items upon the questionnaire of a single 
individual, because the motive for failing to answer is 
simply a matter of conjecture. Silence may mean con- 
sent or its opposite. It may mean ignorance, prejudice, 
laziness, or many other things, and it is usually difficult 
for the instigator to learn which. For these reasons, 
it is extremely important to get a full response. The 
nature and merit of the questionnaire largely deter- 
mines the adequacy of the response, hence the remain- 
ing paragraphs in this chapter are in part answers to 
the present problem. 

3. Presenting the appeal. Whether a questionnaire 
is answered or put in the wastebasket depends largely 
upon the nature of the appeal that is made . There are 
several ways in which to increase the effectiveness of 
the appeal, as follows: (1) By means of the opening 
statements in the questionnaire. These should be of 
such a challenging and interesting nature as to sell the 
investigation and bring forth a response. (2) By 
means of an accompanying letter. Sometimes a spe- 
cial letter is prepared presenting in considerable detail 
the nature of the study, and the reasons why assistance 
is needed. (8) By writing in advance to secure a prom- 


QUESTIONNAIRE TECHNIQUE 179 


ise of assistance. This places the procedure on a much 
more dignified plane than simply sending the blank 
without warning, and in some cases has proved effect- 
ive. (4) By letting each person know that he is one of 
a select list that is being asked to help. It is possible 
to make it clear that his expertness has been discov- 
ered, and to attach a certain amount of honor to 
answering the questions. (5) By multigraphing the 
questionnaires. Printed or mimeographed forms sug- 
gest large-scale operations, and cause each person who 
receives them to feel that his own response is of rela- 
tively small importance. But if the forms are multi- 
graphed, and the person’s address is typewritten with a 
type and ribbon selected to match the multigraphed 
copy exactly, the impression of a personal letter is 
secured, and a greater willingness to respond is thereby 
gained. (6) By presenting the appeal in person, rather 
than by mailing the blanks. It is often better to go 
around to the persons and give them the forms and ask 
their cooperation directly, where the purpose can be 
more adequately explained and emphasized by personal 
appeals. This gives the questionnaire much of the 
value of the interview. (7) By appealing to the per- 
son’s interest in the advancement of science. If it can 
be shown that really valuable results will come from 
the study, no other reason is needed to enlist the co- 
operation of certain types of people. However, this 
is in some cases difficult to show. (8) By offering per- 
sonal rewards. Chief among these is the promise of 
a copy of the returns. It is conceivable, however, that 
financial rewards for cooperation might be justified in 
certain cases. 

4. Getting official support. The most successful use 
of the questionnaire is in those situations where replies 
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can be demanded by officials of some kind. Govern- 
mental authorities can get what facts they need by 
compelling answers. School principals or superintend- 
ents can get 100% replies from the teachers in their 
own schools, if. they so desire. Teachers can get com- 
plete returns on questionnaires from their pupils, if 
they submit them as a regular classroom assignment. 
When the instigator has no such official relation to 
those from whom he desires information, it is some- 
times possible to enlist the support of such an official 
person and work through him. Sometimes the forms 
may be sent out over the signature of such a cooperat- 
ing official, or they may be accompanied by his letter 
of introduction. Again, such an official may simply dis- 
tribute the blanks at a meeting of his subordinates, 
thereby signifying in an unofficial way that he is in 
sympathy with the study. The forms may, in other 
cases, be accompanied by a letter from the faculty 
member who is directing the research. In any case, a 
good rule is that no questionnaire should be sent out 
that has not previously received the approval of some 
person in authority who can judge its merit and sug- 
gest needed revisions. This would greatly reduce the 
number of foolish or useless questionnaires that dis- 
credit scientific work and workers. 

5. Time of sending. There are two important factors 
that affect the time at which a questionnaire should 
be sent. The first is that it should not be sent at a time 
when other interests or responsibilities will interfere 
with it. If, for example, it arrives during a final exam- 
ination week, its chances of being considered are 
greatly reduced. The other factor is that it should 
arrive at a time when the information desired is avail- 
able. If it comes too early, there may be no informa- 
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tion available, or if too late the facts are not fresh in 
the mind and the answers will lack reliability. 

6. Pre-tabulating the replies. The questions should 
not be sent out until the use to which the replies are to 
be put has been clearly determined. This means that 
the tables that are to be made should be decided upon, 
and the various columns and headings actually written 
down on paper beforehand. The investigator’s own 
answers to the questions, real or imaginary, should 
then be recorded in the tables as a sort of rehearsal to 
be sure they are adapted to the replies. If the ques- 
tions and tables are made together, the replies are not 
only more directly applicable, but also more economic- 
ally handled when they come in. Unless this is done, 
the replies will ordinarily have to be edited or inter- 
preted or classified before they can be tabulated, with 
consequent distortion of their original meaning. 

7. Preliminary rehearsing. The interpretation placed 
upon a given question by those who answer it may be 
quite different from that of its author. [If it is tried 
out on several people in a preliminary study, and they 
all understand it correctly, it may be considered as safe, 
but if a few of them misunderstand it or answer it 
from a different viewpoint than that intended, revi- 
sions are needed. To make up a tentative form and try 
it out and revise it before distributing it generally is 
expensive, and sometimes hard on the investigator’s 
patience, but the care is amply repaid in improved re- 
sults. 

8. Number of questions. The short questionnaire is 
much more likely to be answered than the long one. 
Furthermore, each item in it is more likely to be 
answered carefully. It is better to have a few highly 
significant questions than many trivial ones. As a 
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general rule, a little extra time and effort will reveal 
possibilities of omissions and combinations which will 
greatly improve the results. There is an exception to 
this rule, possibly, in the case of official question- 
naires, where answers to long ones can be required as 
well as short ones, but even here the mental attitude 
produced by a burdensome list of questions may be 
so bad as to reduce the quality of the answers. 

9. Classification schemes. Questionnaires fall into 
two general types in regard to the schemes of classi- 
fication of the material in the answers. One type calls 
for the answers according to the system of thought of 
the respondent, letting him write freely according to 
his own natural associative processes. The investi- 
gator then rearranges the information to fit his own 
scheme of thinking, putting the various points or facts 
into such places in the classification scheme as seem 
to be most suitable. The other type forces the re- 
spondent to fit his answers into the predetermined 
moulds or classifications of the investigator. In this 
type, the field is divided up into such sections or parts 
as the investigator sees fit, and this classification is 
imposed upon all those who answer. The advantage 
of the former type is that it does not hamper the 
respondent by forcing him to fit his ideas into a scheme 
of thought that runs “against the grain” of his own 
thinking. On the other hand, it imposes upon the 
investigator an almost hopeless task of making the 
necessary adjustments from the various classification 
schemes to his own, and results in a certain amount of 
distortion of the various persons’ answers when he 
reclassifies them. Unless there is some very good 
reason for doing otherwise, the questionnaire should 
be arranged according to the second type. 
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10. Ambiguity. There are two types of ambiguity 
which interfere with questionnaire studies. The first 
is ambiguity of questions. Unless the questions are 
very carefully stated, and then checked by rehearsal 
with a preliminary group, they are likely to mean 
different things to different persons, producing incom- 
parable and misleading replies. Terms should be 
defined exactly, and in cases where there could be any 
doubt the use of one or more synonyms in the question 
should be resorted to. The questions should be so 
worded as to be understood by the least intelligent 
person who is to answer them, and should be so clear 
to all respondents that all effort will be expended in 
answering them, rather than in interpreting them. 
In so far as the questionnaire is an intelligence test, 
it defeats its purpose. The second type of ambiguity 
that interferes with questionnaires ig ambiguity of 
answers. Not only must the author’s meaning be clear 
to the respondent, but the respondent’s meaning must 
be clear to the author when he attempts to use the 
replies. The questions must be so worded as to call 
forth answers of a type which cannot be misunderstood 
by the person who is to use them. 

11. Objective answers. An excellent device which 
serves a number of different purposes in improving 
questionnaire studies is the use of objective answers, 
or short answers with unmistakable meanings. There 
are several types of these: (1) The “yes or no” 
answers, such as, “Do you give school credit for home 
work?” (2) The quantitative answer, such as, “What 
percent of grade on an examination paper do you base 
on the use of English?” (8) The check answer, 
wherein all possible answers are listed on the sheet 
and the respondent checks those to which he subscribes, 
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(4) The rating answer, wherein each of several items 
is judged to be “much, little, or none” etc. (5) The 
ranking answer, in which the several items listed are 
to be ranked according to the judgment of the respon- 
dent. (6) The test answer, wherein the questionnaire 
is administered as an ordinary new-type test, in which 
the person answering is eager to make a good score. 
An illustration of this is the use of such a true-false — 
test as “Radicals who commit crimes should be 
executed without trial,” instead of ‘‘What is your 
attitude toward the treatment of radical criminals?” 

12 Factual questions. The value of a questionnaire 
is increased if the questions asked are of a factual 
nature. Questions calling for judgment or opinion are 
ordinarily of minor value. The use of the question- 
naire by workers in the early child study movement 
has been seriously criticized because of the emphasis 
placed upon impressions or generalizations about child 
nature rather than upon actual cases of child behavior. 
If the questions bring forth a large number of specific 
facts or incidents or cases, the investigator can gen- 
eralize upon them by the aid of scientific or statistical 
techniques, but if the persons who answer the ques- 
tions report their own unscientific generalizations, the 
investigator can hardly deal with them in a scientific 
way. 

13. The resort to memory. To rely upon the 
memory of the person who is to furnish an answer 
reduces the value of the study. This is especially true 
when the events to be reported by adults are remem- 
bered from childhood. Ags time passes, memory be- 
comes mixed with imagination, and it is almost 
impossible to separate fact from fiction. It is better 
to ask for facts concerning events as they happen than 
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after they have happened, unless, as is sometimes 
the case, the questionnaire is to be answered later from 
records that were actually made at the time of the 
events. For instance, if the purpose of the study 
were to determine the daily activities of a group of 
students for a week, it would be safer to have them 
keep their records daily than to have them write up 
the whole week’s activities at the close of the period. 

14. Impossible questions.. The usual questionnaire 
prepared by untrained investigators contains one or 
more questions which could not possibly be answered 
by the persons to whom it is sent. Such questions 
are either discarded in disgust, or else answered by 
guessing, which leads to false conclusions. There are 
two ways of overcoming this difficulty: (1) Send 
questions only to persons who are known to possess 
the information or to be able to secure it from reliable 
and convenient sources. (2) Limit the questions to 
the kind of information known to be in the possession 
of the persons who are called upon. 

15. Leading questions. The makers of question- 
naires may learn a valuable lesson from the courts, 
in that leading questions distort the truth. Human 
nature is such that suggestion plays a strong part in 
determining the answers to questions. The way a 
question is asked has much to do with the answer 
received. If the investigator has a bias, he must be 
careful not to show it, else he will get a larger number 
of answers from persons who share his bias, and this 
will influence the kind of answers received from 
persons who are actually neutral. 

16. Anonymous replies. Sometimes the chief prob- 
lem in a questionnaire study is to get replies which 
are truthful. Questions calling for personal informa- 
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tion that might be embarrassing are of doubtful value 
unless they are answered anonymously. Students who 
are asked to report their study practices or their 
reactions to courses are seldom content with the 
assurance that their grades will not be influenced by 
the answers. Even leaving off names is not always 
enough since there is a fear that the teacher will 
recognize the handwriting. On the other hand, there 
are situations in which better replies are given when 
names are attached. Students who hide behind anony- 
- mous replies to be smart and clever are more likely 
to take the task seriously if obliged to sign their 
names. Also the names enable the investigator to 
correlate the returns with information from other 
sources, as intelligence tests, school grades, etc. 

17. Spatial arrangement. The physical arrangement 
of the questions on the page is of some importance. 
Merely providing a line or a hollow square or paren- 
theses for the answer makes it possible to get all 
answers uniformly arranged, and thus aids tabulation. 
The arrangement also has something to do with the 
omission or overlooking of questions. Also the amount 
of space allowed for answers influences the kind of 
answers received. For example, if short objective 
answers or strictly categorical yes or no answers are 
desired, it is well to crowd the questions so that no 
room is allowed to write long qualifying statements. 
On the other hand, if a question calls for a long state- 
ment, but has only a small space, a good opportunity 
has been lost. A large space encourages a long 
answer, and a short space the opposite, even though 
the directions state that supplementary sheets may 
be used if necessary. 
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EXERCISES AND QUESTIONS 


1. Why is the questionnaire in such general disrepute as a re- 
search device? How may this situation be improved? 


2. Under what circumstances is the resort to the questionnaire 
desirable? When not? 


8. Make a list of problems which seem to require the use of the 
questionnaire. Examine each to see if some other method 
of attack would be feasable. 


4. Actually draft a questionnaire for use with one of the prob- 
lems listed in the previous exercise, and submit it for 
criticism. 

5. Prepare a brief questionnaire to be answered by your fellow 
students. Check up the results and see how many of the | 
questions actually achieved their purpose. 


6. Examine the questionnaires in one or more research reports 
and criticize each as to its merits and demerits. 


7. Read the discussion for and against the use of the ques- 
tionnaire and formulate what you consider a tenable 
position regarding it. 

8. Classify questionnaires according to their types. According 
to the purposes which they are to serve. 
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CHAPTER XI 
OBSERVATION TECHNIQUE 


1. Active versus passive observation. The process 
of scientific observation is quite different from merely 
“looking around.” It involves a study procedure rather 
than a mere visiting procedure. It requires methodical 
and painstaking work rather than a mere tickling of 
the senses. It is often necessary to continue a task 
for a considerable time after it has ceased to be 
interesting. The procedure which yields the greatest 
return in interest or satisfaction, or even in valuable 
personal experience to the observer, is not necessarily 
the one which will make the greatest contribution to 
educational science. 

2. Number of specimens. Since uniformity of speci- 
mens is almost never to be found in educational phe- 
nomena, observations must be made upon a number 
of different cases in order to discover what is typical. 
Observations upon several school buildings, several 
English classes, or several instances of misconduct will 
yield considerably richer data than observations upon 
only one case. As a rough working rule, it is fairly 
safe to say that by the time thirty specimens of a 
given type have been observed, the additional ones 
cease to contribute an appreciable number of new 
elements. By comparing the records of the various 
observations, however, it is comparatively easy to 
discover when the point of diminishing returns has 
been reached to such an extent as to make further 
observations unprofitable. If the various specimens 
which are observed are compared one with another, 
the individual traits of each and the universal traits 
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of all will be more readily discovered, thus leading to 
more helpful generalizations. Probably scientific ob- 
servations more often suffer from the comparison of 
an insufficient number of specimens than from too 
many. 

3. Sampling. The time and labor involved in obser- 
vation make it impossible to observe all specimens or 
cases of a given phenomenon; consequently it is 
necessary to resort to sampling. It is better to select 
a few typical cases for careful and thorough observa- 
tion than to make superficial observations of a large 
number. The important essential, here as elsewhere, 
is that the cases chosen should be representative of all 
the cases from which they are selected. If they are 
selected in a haphazard way or according to the whim 
of the observer, they may yield an entirely false 
impression of the facts. For instance, if the best 
schools, teachers, or pupils are chosen, the results are 
misleading in one way, and if the worst ones are chosen 
the error is in the other direction. 

4, Repeated observations. There is much to be 
gained from observing the same specimen more than 
once. A second look is almost certain to reveal things 
not seen the first time. The first time may be a special 
case, an “off day” possibly—but several repetitions 
will undoubtedly yield a more reliable impression. 
Supervisors have learned that it is unwise to base 
their estimates of a teacher’s efficiency upon one visit 
to her classroom. Furthermore, there is a limit to 
what can be seen and appreciated at one time, some- 
what as there is a limit to the amount that can be 
eaten at one meal. To observe and go away to study 
over the notes taken in order to “digest” the 
experiences gained makes it possible to return with 
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a number of new questions, or with a renewed appetite 
for knowledge. It should be kept in mind, however, 
that later observations depend for their value upon 
their progressive nature. That is, each should be 
built upon the results of the previous ones, so that 
the whole series is cumulative. 

5. Duration. Hasty observations are likely to miss 
the big things. The true scientist considers his time 
of no value so long as there is chance of out-waiting 
a phenomenon and thus forcing it to come out into 
the open where it can be seen. Numbers of conclusions 
in science have been proven false because of factors 
which developed after the close of the observation 
period. For example, a very short period for the 
observation of a patient who has been administered 
a stimulant might create the impression that he is 
rapidly recovering, but a longer period would reveal 
quite the opposite. Similarly, educational experiments 
which are yet in their youth are sometimes misjudged 
because of the lack of time for all aspects to reveal 
themselves. Watson has called attention to the need 
of continuous observation of infants from the hour 
of the birth on as the only sure way to discover what 
forms of behavior are really instinctive and what forms 
are learned through experience. 

6. Previous preparation. It takes time and serious 
study to get ready for scientific observation of a 
phenomenon. It is necessary to know the general field 
through and through in order to know what to look 
for. The more one knows before the observation, the 
more he is likely to learn during it. Professional 
reading on the subject of school buildings or methods 
of teaching is essential as a preparation for visits to 
school plants or classroom recitations. This reading 
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and study might well look toward the following objec- 
tives. (1) Mastery of the general technique of obser- 
vation, as formulated in the books on observation of 
teaching. (2) Discovery of the standards of merit 
which are to be used in judging the particular phenon- 
enon or thing to be observed. (3) Planning a cam- 
paign or schedule of procedure to be followed in the 
actual observation. 

7. Observation guides. The use of a list of points 
to look for makes it possible to learn more about what 
is observed. Unless certain definite objectives are 
kept in mind, it is entirely possible for more interesting 
but unimportant matters to crowd the real essentials 
out of the attention. It is not uncommon for irrelevant 
events adjacent to or concurrent with the things being 
observed to gain more attention from the observer than 
the thing itself. A written or printed list of items to 
be rated or recorded, or a blank form to be filled in 
with the results of the observation will usually more 
than repay the effort required to make it. 

8. Some things to look for. What to look for 
naturally varies according to the problem, but some 
idea of the infinite possibilities that are before the 
observer may be gained from the following suggested 
list of items: (1) The objects involved, as glass, 
wood, people, paper, etc. (2) The attributes or quali- 
ties of these things, as blue, hard, tall, ete. (3) The 
actions involved. (4) The locations or spatial relations 
of the things. (5) The duration, or time, of the events. 
(6) The functions or purposes of the various items. 
(7) The factors responsible for the phenomena or 
events. (8) The cost. (9) How they work. (10) The 
environment or setting in which they are located. (11) 
The probable future or outcome of events. (12) The 
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classification schemes into which they fall. (18) Dif- 
ferences between the various specimens. (14) Differ- 
ences between these and previously observed cases. 
(15) Facts not obsrved before. (16) Inferences from 
what is observed, as to their significance for other 
facts. 

9. Objectivity. A little time spent in translating 
subjective matters into objective form that can be 
definitely formulated and recorded adds enormously 
to the scientific value of observations. It is well to 
make conscious effort beforehand to list the tangible 
or objective items in the given situation which lend 
themselves to observation. For example, ‘Teacher 
domination of the recitation” is a mere matter of 
subjective judgment or estimate until a little study 
suggests the use of a stop watch to record the portion 
of the hour during which the teacher talks and the 
portion during which the children talk. Similar devices 
may be employed on other types of observation with 
similar improvement in the results. 

10. Instruments of precision. It has long been 
known that there are rays of light which can not be 
seen by human eyes, and sounds that are of too high 
or too low pitch to be recorded by human ears. Sim- 
ilarly, there are objects too small or too far away to 
be seen without the aid of microscopes or telescopes. 
For many purposes, therefore, it is necessary to use 
mechanical extensions or supplements to the sense 
organs in order to improve observations. Sometimes 
the unaided senses may be useful in exploring the 
field and in locating problems, after which instruments 
are needed to study them in detail. In some cases 
the necessary instruments are already available, but 
in other cases the scientist must invent or devise them 
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especially for his purpose, as was done in the case of 
the machine for photographing eye movements in 
reading. The stop watch, the motion picture machine, 
the ordinary camera, the X-rays, the use of films 
sensitive to ultra-violet or infra-red rays, the tono- 
scope, the microscope, the thermometer, standardized 
tests, etc., are illustrations of instruments that have 
been useful in making observations of educational 
phenomena. It should be said, however, that too much 
precision is as bad as too little. Some things can not 
be observed as well through the microscope as through 
the naked eye because of the details obscuring the 
main features. 

11. Measurements. Measurement is simply a refined 
form of observation. It attempts not simply to deter- 
mine the presence or absence of a phenomenon or 
practice, but also its degree or extent. It seeks not 
for a “yes or no” answer but for an answer to the 
question “how much.” If observations are reduced to 
quantitative or numerical terms, they become more 
precise and lend themselves to more refined statistical 
treatment. It is almost always preferable to sum- 
marize the results in numbers rather than in adjectives. 
Many facts are more adequately studied by the use 
of objective tests than by mere subjective estimates. 
For example, if it is desired to learn how well students 
have learned library technique it may be better to 
give a true-false test made up of such items as the 
“The Readers’ Guide is an index to periodical litera- 
true” than to spend several days watching students 
work in the library. This does not mean that subjec- 
tive estimates should never be employed, but rather 
that they should be used only to supplement the use 
of improved forms of measurements. 
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12. Experimental control. It is sometimes necessary 
to provide special laboratory conditions where a phe- 
nomenon may be observed under more favorable and 
more carefully controlled conditions. For example, the 
horses of Elberfeld were observed to do such remark- 
able things as answering questions in several different 
languages or computing difficult problems in mathe- 
matics, and apparently possessed most unusual intelli- 
gence. But when the psychologists took the trouble 
to control the conditions incident to these unusual 
performances, they discovered that the horses were 
getting signals from persons in the audience and that 
their supposed intelligence did not exist. If a phenom- 
enon can be observed under conditions which free it 
from irrelevant and confusing factors, or if it can 
be observed, the conditions varied, and then observed 
again to note the change, the truth is more likely to 
be discovered. Sometimes the setting is more im- 
portant than the thing itself, and if the setting can 
not be controlled, it is impossible to know how much 
to attribute to it. Again, it is important sometimes 
to be sure that the setting remains constant through- 
out the period of observation, else a shifting of condi- 
tions may be confused with the phenomenon itself. 
In short, observation is best performed in the exper- 
imental laboratory. 

13. Typical conditions. A caution is needed at this 
point lest the last paragraph leave a wrong impression. 
In the effort to control conditions for observation, care 
is necessary to avoid creating an artificial or unnatural 
situation that is not typical of ordinary life conditions. 
For example, a child’s reading when his head is 
strapped into position for his eye movements to be 
photographed may not be entirely the same as when 
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he is reading a story for his own amusement. The 
mere presence of the observer in a classroom may dis- 
tract the attention of the pupils or make the teacher 
nervous. The nature of the thing to be observed has 
much to do with whether to set up controlled laboratory 
conditions or to go and see it in its natural setting, 
but whichever method is used, it is important that an 
unnatural situation should be avoided as much as 
possible. 

14. Abnormal cases. The average man knows more 
about automobiles from what he has picked up when 
they were broken down or functioning badly than from 
his experience with them when going smoothly. The 
abnormal cases in educational affairs may shed con- 
siderable light on the normal case. The science of 
medicine owes much to what has been revealed by 
disease, and psychology owes much to abnormal psy- 
chology. Frequently the abnormal case is not essen- 
tially different from the normal, being simply an 
unusually emphatic form which thereby attracts 
attention. The point is not that we should devote all 
or even a major part of our energies to problem child- 
ren or abnormal cases, but rather that we should go 
to these for the light which they may shed on the 
whole situation. 

15. Records. The major value of observations is 
lost unless suitable records are made. The principal 
points to be considered in this connection are as 
follows. (1) The records should be complete and 
detailed, not vague generalities or mere impressions. 
(2) They should ordinarily be made upon previously 
prepared blanks, tables, charts, or forms, with a 
regular place for each item to be recorded. (3) In 
most cases the records should be made at the time of 
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the observations, while the impressions are vividly 
stamped upon the senses, although there are situations 
in which the writing of notes would interfere with 
observation or possibly with the normal behavior of 
the process being observed. In this case the records 
should be made immediately after the close of the 
observation. (4) In some cases it is better to be 
receptive to what is shown rather than to take the 
initiative in trying to discover things. For example, 
the guide that shows a group through a factory prob- 
ably has a better prepared plan for the excursion than 
the guests who are going through. In such a case, 
it may be better to take the information in the form 
in which it is supplied and to organize the impressions 
later according to the plans for the particular study. 
(5) The recording of observations is not only to be 
used for the sake of the resulting record, but also as 
an aid to the observation itself. To draw what is 
seen, or to describe it in writing, directs attention to 
the essentials and enables the eye to grasp more of 
what is before it. Each attempt to draw or to describe 
improves the succeeding effort to observe. (6) Such 
papers, materials, samples, or parts of the thing 
observed as can be carried away feasibly should be 
made a part of the record. These include photographs, 
souvenirs, printed schedules, bulletins, mimeographed 
lesson materials, lists of articles used, specimens of 
student work, etc. Such articles give reality to the 
records and enable the analysis of the situation to 
continue after the close of the observation period 
proper. 

16. Later study. Just as reading beforehand about 
the thing to be observed enriches the ensuing obser- 
vation, so does reading about it afterwards add to the 
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interpretation of the impressions gained. Teacher- 
training institutions have learned that the value of 
classroom observations is greatly increased if there 
are follow-up lessons and group conferences based on 
them. An alternation of study and observation periods 
seems to be the most effective arrangement for 
discovering the full meaning of the things observed. 

17. Supplementary methods. Observations yield 
richer returns when combined with other supplemen- 
tary techniques. For example, the interview will pro- 
duce many items of information which mere obser- 
vation alone will not give, as in the case of having 
a talk with a teacher about her classroom technique 
before or after observing a recitation. The values of 
the experimental method, and of measurement as aids 
to observation, have already been mentioned. ‘The 
historical technique is sometimes important, since the 
previous history of the thing observed is essential 
to its effective interpretation. Statistical techniques 
are needed to treat the results of the observations. 
The case study technique has its special contributions, 
as does also the method of introspection, or observing 
one’s own behavior, in certain cases. 

18. Ethics and diplomacy. There are several little 
matters which an investigator using the observation 
technique may well keep in mind to preserve the right 
state of mind on the part of those with whom he has 
personal relations. Some of these are as follows: (1) 
Be careful not to disrupt the classroom work because 
of the observation. One good way to avoid this is to 
enter the room before a class begins and remain until 
after it ends. (2) Do not manifest approval or dis- 
approval of what is seen, lest this cause a change in 
the procedure or create an undesirable state of mind 
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on the part of the persons involved. (3) Make a 
good impression upon the persons concerned so that 
a way will be left open for later observations. (4) 
Refrain from telling the parties observed how others 
perform the operation. The purpose of observations 
is learning and not teaching. New ideas should always 
be welcomed by the observer, but not imparted by 
him to the persons observed. (5) In some cases note- 
taking should be reduced to a minimum or postponed 
entirely until after the observation is completed, lest 
it produce the wrong effect upon the persons observed. 
It is well to take a position from which the notes which 
are written can not be read by the persons whose 
reactions are described, in order to get greater free- 
dom to write down the facts without restraint. (6) 
Do not tell tales about the person observed, even if 
they are true. The observer is the guest of those 
observed, and to make remarks about the host after 
partaking of his hospitality is obviously unkind. 


EXERCISES AND QUESTIONS 


1. How important or valuable do you consider observation as a 
research technique? Why? 


2. Wherein is observation superior to interviews and question- 
naires? Wherein inferior? 


8. How can ability to observe be improved? 


4, In what sense is observation an essential part of every 
scientific investigation? 


5. List a number of research problems of your own school 
which lend themselves well to the use of the observation 
technique? 


6. Prepare a list of elements to look for in connection with one 
of the problems mentioned above. 


7. What are the essentials of successful scientific observation? 


8. Explain why more research workers do not resort to visita- 
tion and observation instead of sending out questionnaires. 
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CHAPTER XII 
MEASUREMENT TECHNIQUE 


1. Providing a measurement situation. Measuring 
is more than simply giving tests. It frequently hap- 
pens that the main task is to prepare such a situation 
or lay such a foundation that the tests when given 
will tell what it is desired to know. For instance, if 
the problem is to measure the results of a year’s work 
in arithmetic, it is necessary to prepare for the final 
measurement by giving a test at the beginning of the 
year to serve as a basis for comparison. An enormous 
amount of waste takes place annually in the schools 
because of indiscriminate giving of tests which by the 
very nature of the situation could not yield very useful 
knowledge. Sometimes an experimental procedure is 
necessary as a preparation for the measurement. In 
short, measurement serves its highest purpose when 
it is used as a tool in connection with some other type 
of research procedure rather than ag a thing in itself 
or standing alone. 

2. Time of measuring. There are two common poli- 
cies employed in measuring the results of research 
projects, one the immediate, and the other the more 
remote. For example, a test may be given at the 
conclusion of each lesson or study period to measure 
directly the fruits of that particular unit of work, or 
a whole series of lessons may be taught without any 
testing and the whole measured by a test at the end. 
The advantage of the more remote type of measure- 
ment is that it records the changes or results which 
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are of a more permanent nature and ignores those that 
do not last long after the original exposure. The dis- 
advantage of the remote measurement is that it meas- 
ures the ability in question after it has become mixed 
up with the results of hundreds or thousands of other 
experiences acquired both in and out of school. In 
other words, the longer the time before the measure- 
ment the more the quality or ability measured becomes 
diluted by the addition of other qualities or abilities. 
If it is such an ability ag calls for extremely delicate 
or precise measurement, it is frequently impossible 
to measure it except in its more concentrated form, 
that is, immediately after it has come into existence. 

3. Sampling. A good rule to follow is to measure 
comprehensively; that is, to measure a rather large 
sampling or the whole rather than a small portion of it. 
For ordinary purposes it is easier to estimate a fairly 
large unit of a given ability than a very small one. 
For example, it may be easier to judge a student’s 
collateral reading results by the term paper he pre- 
pares after weeks of work than by the test on one 
day’s reading. When the unit of sampling measured is 
microscopic, the measuring instrument must be very 
precise, more precise than yet exist in most types of 
educational work. Not only does the larger sampling 
eliminate the seriousness of small errors in the meas- 
urement, but it also increases the chances of being 
truly representative of the total range of abilities 
sampled. An examination consisting of ten questions 
is much more desirable than one which includes only 
two, but not so good as one of fifty or a hundred. 

4. Many-sided measurement. To measure only the 
length of a room and ignore its height and width is 
no more serious an error than to measure educational 


MEASUREMENT TECHNIQUE 203 


products by one kind of test and ignore the other 
phases which it does not take into account. A new- 
type test may reveal certain things not shown by the 
older essay-type test. A standardized test might show 
still different things. One plan of scoring may not 
take into account the elements which are recognized 
by a different one, as in the case of scoring arithmetic 
tests for correctness of principle and for correctness 
of answer, respectively. A one-sided measurement of 
results of an experiment may reveal startling growth 
in a certain ability when a more complete or all-round 
measuring program would show this to be accomplished 
at the price of a great loss in some other ability which 
had previously been left unmeasured. A safe rule to 
employ is that of using several different kinds of tests, 
and those of as widely different types as possible. 

5. Measurement by analogy. Measurements of abil- 
ities in terms of normal experience are preferable to 
measurements in terms of abstractions from that 
experience. For example, business ability is better 
measured by achievement in business than by tests 
in arithmetic or bookkeeping. Likewise, ability to 
swim is better measured in the water than in a lab- 
oratory where measurements are made of chest expan- 
sion or strength of the muscles. There is a constant 
temptation to resort to measuring by analogy, that 
is, Measuring something that seems to be similar to 
the desired ability but which is more tangible or more 
easily measured, and then to let this measurement 
take the place of the genuine one. Probably the most 
common variety of this is measuring verbal or lin- 
guistic behavior and letting it stand for behavior in 
general. What one says are his attitudes may not 
be at all representative of his real attitudes, either 
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because he is dishonest or self-deceived. Again, a 
person may be quite skillful in a given operation and 
lack linguistic ability to make a passing grade in a 
verbal test upon it, while others can tell how to do it 
remarkably well but are unable to do it themselves 
at all. If measurement by analogy is employed, it 
is extremely important that the analogy actually hold 
true throughout. 

6. Selecting available tests. The number and var- 
iety of measuring instruments for educational work 
has so greatly increased in recent years that it is 
difficult to know what is the best tool for a given 
situation or need. The careful scientist will find it 
necessary to make considerable study simply of the 
various tests, scales, score cards, and instruments of 
precision that others have devised, in order to be able- 
to make an intelligent choice. A course in tests and 
measurements should be a part of the preparation of 
the research worker so that he can make more intel- 
ligent selections of the tests to use. Lacking such 
a training, the least he can do is to secure the descrip- 
tive catalogs of tests published by the three leading 
publishers and study them carefully to familiarize 
himself with the available supply.* 

7. Devising new tests. Many research problems 
in education call for tools not yet invented. The 
investigator must stop and create a test or measuring 
instrument for a specific purpose. The making of 
tests is a large field which cannot be entered here, and 
which should be studied by means of regular books 


*These publishers are: 
(1) World Book Co., Yonkers-on-Hudson, N. Y. 
(2) Public School Publishing Co., Bloomington, Ill. 
(3) Bureau of Publications, Teachers College, Columbia 
University, New York, N. Y. 
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and courses on the subject. The practical research 
worker, however, may find the making of a home-made 
test somewhat easier if he will become familiar with 
those already in existence. A thorough study of these, 
coupled with a close analysis of the particular prob- 
lem, should result in suggestions for a new instrument 
to meet the needs. 

8. Determining the value of tests. An important 
research problem in the field of measurement is that 
of evaluating tests to see how well they actually 
measure and comparing the relative merits of two or 
more different tests. The chief essential for such 
work is a known criterion against which to compare 
the test scores. This is often exceedingly difficult to 
obtain. The following suggested procedure attempts 
to overcome this difficulty. The steps or elements are 
as follows: (1) The basic consideration is that the 
situation is to be controlled, by experimental means 
if necessary, so that it is known beyond a doubt that 
there is a difference between two groups in the trait 
or ability to be measured. For example, one class of 
students may be taught a series of lessons on the civil 
war and another not exposed at all to that instruction, 
thus making certain that there is a real difference 
before applying tests which are to measure it; or one 
class may be given thirty minutes to study a lesson 
while the other class is allowed only five minutes to 
form a rough acquaintance with it before the tests are 
to be administered. For convenience those who are 
known to possess the desired ability will be called 
“sheep” and those who are known not to have that 
knowledge will be called “goats.” The merit of a 
test will then be judged by how well it separates the 
sheep from the goats. (2) Having provided two con- 
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trasted groups, the next step is to administer to each 
group the test or tests to be evaluated. If the problem 
were to see how well a completion test measured, it 
would be given to both groups and the scores compared 
to see how many of the goats were able to get in with 
the sheep, or in other words, how much overlapping 
of scores existed. Or if the problem were to see 
whether the completion test measured better than the 
true-false test, they would both be given to both groups 
and the overlapping of the sheep and goat scores on 
one test compared with the overlapping of scores for 
the same two groups on the other test. If, for example, 
40% of the goats got scores higher than the poorest 
sheep on the true-false test, but only 10% did so on 
the completion test, we should say the completion test 
was a better measuring instrument because it detected 
the known difference between the two groups more 
thoroughly. (3) The amount of overlapping of the 
scores for sheep and goats on the tests may be 
determined with accuracy by any of several methods. 
The frequency curves for each group may be drawn 
and the overlapping observed graphically. The num- 
ber of goats exceeding the lower quartile, median, or 
ten percentile for sheep may be determined, or the 
mean scores of the goats may be subtracted from that 
of the sheep, and the reliability of this difference com- 
puted by getting the experimental coefficient. The 
experimental coefficient for the completion test may 
then be compared with that for the true-false test, 
the larger coefficient indicating the better test. (4) 
In cases where experimental control is impossible, 
resort may be made to selection of extremes that are 
obviously far apart in ability as a means of providing 
the sheep and goat groups. For example, one group 
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may be made up of “A” students and another of failing 
students. Or if the tests to be evaluated are tests 
of character traits, a normal group (sheep) may be 
compared with a group of delinquents (goats) in a 
special institution.* 

9. Precision. A test should measure with that 
degree of precision which is appropriate to the product 
measured. Areas of counties are measured in square 
miles and not in square inches, but precious stones 
are measured in carats rather than in pounds. Num- 
ber is the greatest contributor to precision. Measure- 
ments that result only in adjectives, as “large,” “‘good” 
or “valuable,” are ordinarily far from precise. <A test 
should yield numerical scores, and these in small 
enough units to produce a sufficient spreading out or 
distribution of the cases measured to enable them 
to be handled readily. Thus a test which only sorts 
the students into two classes, “pass” and ‘‘fail,’”’ leaves 
too many people together with the same score, while 
a five-grade system distributes them more. A test 
should be so easy that no one fails to do something 
on it, and so difficult that no one makes a perfect 
score, and should result in a sufficiently large range 
of distribution to make the cases stand out fairly well 
from one another. 

10. Diagnostic tests. To know a child’s total score 
on an arithmetic test is of some importance, but to 
know separately his scores on each of the processes, 


*For further details and examples of this technique see: 


(1) Crawford, C. C. and Raynaldo, D. “Some Experi- 
mental Comparisons of True-False Tests and Tradi- 
tional Examinations”; in School Review, Vol. XXXIII, 
Nov. 1925; 698-706. 

(2) Crawford, C. C. “Of What Value are True-False 
Tests”; Educational Research Bulletin of the Los An- 
geles City Schools, Vol. VI, Feb. 1927; 3-6. 
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as addition, subtraction, etc., is much more important. 
The idea of a diagnostic test is to get such analytical 
results, or to discover the nature as well as the 
amount of the abilities studied. The research worker 
should be encouraged to use this type of measurement 
more, even if he wishes to combine the parts and get 
a composite score. The old saying, “you can’t un- 
scramble eggs,” suggests that it is better to make 
separate records for the various parts of a test, and 
later combine these, than to attempt to analyze later 
what has been measured or recorded only in composite 
form. 

11. Uniform gradations. If scores on a test are 
to have maximum value, the units should be of the 
same size at all points in the scale. Such is seldom 
the case. For example, the difference between a 
grade of 25% and 35% on a test is of small importance 
as compared with that between 65% and 75%. Like- 
wise, a gain of 10 words a minute in typewriting in 
the first month of practice is not the same as 10 words 
gained after reaching a speed of 100 words per minute, 
because progress in the early period is easier to make. 
Educational products should be measured in a manner 
comparable to the measurement of gases, taking great 
care that measurements be made under uniform 
degrees of density or pressure. The experimenter 
must beware of comparing the improvement made by 
advanced students with that made by beginners, since 
the advanced students may be very close to the limit 
of possibility, requiring greater effort to produce equal 
results. The device for transmuting test scores into 
units which are equal at all points in the range is 
called “scaling.” There are various methods of doing 
this, among the most common of which is the T-score 
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method. These are described in the books on tests 
and measurements. 

12. Directions. There are two major essentials of 
good directions for a test. The first is that they should 
be clear enough to permit those taking the test to 
devote their attention to taking it rather than to 
finding out how to take it. Incomplete or indefinite 
directions may make the test a measure of intelligence 
or previous experience with tests rather than of the 
ability which it is desired to measure. The other 
essential is that the directions should cause all persons 
to react to the test in the same or uniform way. They 
must be understood alike by all candidates, else the 
results are not comparable. Also the directions must 
equalize speed, interest, etc., on the part of the candi- 
dates so that one will not work at a furious pace while 
another proceeds in a leisurely way. McCall suggests 
that it is better to equalize such things by having 
all work at maximum rather than at average pace. 
For example, he would have them all work “as rapidly 
as you can” instead of “at your usual rate.” 

13. Validity. A test is valid when it actually meas- 
ures what it pretends to measure. There is constant 
need for examining the tests used upon this point. 
Many elaborate investigations have been proved worth- 
less because it was found later that the tests used 
were not valid. For example, the aim may be to 
measure knowledge when in reality the main thing 
measured is speed, or ability to write good English, or 
ability to use the teachers pet terminology, etc. A 
certain person of unquestioned ability made a score 
on an intelligence test that indicated feeble-minded- 
ness, simply because his past training had emphasized 
extreme carefulness and double-checking his work for 
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accuracy. His low score was, therefore, not vaild 
because it was due to carefulness rather than low 
intelligence. | 

14. Reliability. A test is reliable if it yields the 
same result every time it is given. If, however, a 
given pupil’s score is high at one time and low at 
another when the ability measured remains constant, 
the test would be classed as unreliable. There are two 
things which the research worker who uses tests 
should do about reliability. (1) He should determine 
how reliable his tests actually are. This may be done 
by several methods. One is to correlate the score on 
the even items with those on the odd. Another is to 
give the test twice and correlate the resulting scores. 
Or, if he wishes, he may compute the reliability coeffi- 
cient by statistical methods. (2) He should increase 
the reliability of his tests. This may be done by 
making it longer, as a large sampling yields more 
stable results than a small one. A test of 100 items 
is more reliable than one of 50 items. Giving the test 
more times, or giving it to more persons, also increases 
reliability. 

15. Standards. In olden days it is said that a “foot” 
meant the length of the foot of the king or chief, 
and would therefore vary from time to time and 
from country to country. The units for most educa- 
tional measurements are not even that well standard- 
ized. A child’s intelligence or knowledge of a subject 
is still measured by comparison with some imaginary 
standard in the teacher’s mind which fluctuates over 
a wide range even in the course of grading a single 
set of papers. The remedy for this is standardization, 
or the development of norms. Specimens of work or 
achievement may be set up as standards and all work 
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compared with these, as in the case of the handwriting 
scale, or a test may be given to large numbers of 
children, their scores computed, and these results used 
as a standard with which to compare individual cases. 
The research worker finds it very advantageous to use 
tests or scales which have previously-established norms 
wherever possible, and, in the absence of such a situa- 
tion, he sometimes has to use the average of his own 
group as a temporary or tentative norm. 

16. Objectivity. A measurement is said to be ob- 
jective if it may be made by different people without 
changing the results. In other words, it should not 
depend on personal opinion of the investigator. The 
new-type test is more objective than the former essay 
type. Standard tests are more objective than teachers’ 
judgments. The research worker should study care- 
fully the recently developed literature on the objective 
or new-type examination to familiarize himself with 
the multitude of practical devices for measuring in 
objective ways things which have for so long been con- 
sidered as mere matters of subjective judgment.* 

17. Subjective judgments. Since perfect objectivity 
is so often impossible to gain, resort must constantly 
be made to estimates and ratings of a subjective na- 
ture. To discard such forms of measurement at the 


*A few of these books are as follows: 


(1) Paterson, D. G., Preparation and Use of New-Type 
Examinations, World Book Co. , Yonkers, N. Y. 

(2) Ruch, G. M. , Improvement of. the Written Examina- 
tion, Scott, Foresman & Co., Chicago. 

(3) Russell, Chas., Classroom Tests, Ginn & Co., Boston. 

(4) Ruch, {e M.. and others, Objective Examination 
Methods in the Social Studies, Scott, Foresman & Co., 
Chicago. 

(5) Weideman, C. C., How to Construct the True-False 
Examination. Teachers College, Columbia Univer- 
sity, N: Y. 
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present stage in the development of educational science 
would be folly. The following methods, devices, and 
suggestions may help to improve the quality of this 
type of measurement: (1) Analysis into smaller 
units. When a complex object, process, or ability is 
broken up into sufficiently small divisions, each division 
can be judged with a high degree of precision. This 
is done by expert stock or poultry judges, who score a 
single part of the animal at a time instead of the whole 
animal at once. (2) Use of rating scales and score 
cards. For many educational purposes, scales and 
score cards are already available. They consist of lists 
of qualities to be judged, and sometimes also involve 
some kind of suggested weights for each, such as dis- 
tributing 100 possible points among eight or ten differ- 
ent qualities. (8) Ranking. Where there are several 
persons, objects, or specimens to be judged, simply 
arranging them in the order of merit is sometimes a 
quite definite and satisfactory form of measurement. 
(4) Comparing all cases with one or more cases which 
have been taken as standards. An example of this 
method is a study that involved rating the value of 
school subjects by comparing them with English, which 
_was given a value of 10 points arbitrarily to serve as a 
starting point. (5) Practicing the making of the par- 
ticular type of judgments. Persons who first begin 
judging handwriting by a regular handwriting scale 
find that it involves an element of guessing at first but 
that, as more cases are scored, the judgments become 
more refined and the results more reliable. Compari- 
son of one’s own ratings at different times facilitates 
this improvement. (6) Making several different esti- 
mates of the same thing. Several independent ob- 
servations and ratings of an object are more likely to 
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average the true result than is one single rating. (7) 
Having several persons make judgments. The con- 
census of several persons’ judgment is more reliable 
than a single person’s judgment, provided that the 
judges are all competent, since personal prejudices may 
be supposed to neutralize each other somewhat. (8) 
Using code numbers or other devices to eliminate per- 
sonal prejudice. Sometimes it is impossible to com- 
pare two sets of student papers fairly unless they are 
freed from all trace of their group membership by the 
use of some sort of key numbers until after the ratings 
have been made. (9) Having persons do the judging 
who are not familiar with the object of the study. The 
experimenter may be unable to compare the two groups 
fairly but an outsider who did not know what the ex- 
perimental variables were would have no such preju- 
dices. 

18. Treatment of results. Measurement is not an 
end in itself. It is simply a means to some other 
worth-while end. The value of the most careful and 
painstaking measurements imaginable may be nullified 
by crude or improper statistical treatment of the re- 
sults. There is as great need for precise ways of treat- 
ing data secured from measurement as for precise 
measurements themselves. The scores tell nothing in 
themselves. They have to be treated in such a way 
as to cause them to tell their story. For this reason 
the present chapter should be supplemented by further 
study about statistical technique. 
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EXERCISES AND QUESTIONS 


1. Show how measurement is necessarily a feature of all re- 
search. 

2. Criticize the statement that the science of education is due 
largely to the measurement movement. 

8. What are the essential requirements of a good measuring 
instrument? 

4, Distinguish between research upon tests and research by 
means of tests. 

5. Make a list of research problems upon tests which would be 
of especial benefit to your school, and indicate how you 
would attack each. 

6. Do a similar thing for research problems to be attacked 
by means of tests. 

7. Read and criticize a number of measurement investigations 
as to their technique. 

8. What difference in knowledge of measurement technique 
would you think there should be between the research 
worker and the classroom teacher who uses tests? Why? 
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CHAPTER XIII 
STATISTICAL TECHNIQUE 


1. The statistical viewpoint. Statistical processes 
and methods are used in such a wide variety of situa- 
tions that a single statement of the nature of the sta- 
tistical viewpoint is difficult to make. The following 
points will shed some light upon it, however: (1) It 
is quantitative. Facts must be stated in numerical 
form in order to be treated statistically. (2) It is 
founded upon the concept of individual differences. It 
is of little importance in chemistry or physics because 
a given object or chemical element behaves the same 
as all others of its class. But biological, psychological, 
and sociological phenomena do not possess such uni- 
formity and therefore necessitate special technique for 
discovering the general trend running through a num- 
ber of widely varying individual cases. (3) It takes 
things as they are and classifies them to make them 
yield desired information, whereas the experimental 
method employs the artificial control of things to dis- 
cover the truth about them. Consequently, the statis- 
tical method is especially good as a substitute for the 
experimental method on such problems as do not per- 
mit of controlled conditions. (4) It is based on sam- 
pling. Hardly ever does the statistician have data on 
all cases which he is studying. He usually works with 
facts about a selected few which are known or sup- 
posed to be a typical sampling from the total group. 
He observes this sampling and derives from it some 
kind of statistical formula for predicting how the whole 
mass would also behave. (5) It looks to what is com- 
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mon rather than to what is unique. It seeks to derive 
from a number of cases a general rule, law, or formula 
which expresses a universal relationship that holds 
true for all cases of that kind. In other words, it is a 
refined instrument for arriving at inductive conclu- 
sions. 

2. Limitations of statistics. It has been said that 
“figures can’t lie, but liars can figure.” Also that there 
are three kinds of liars, “liars, damn liars, and statisti- 
cians.” It has further been often said that the science 
of statistics suffers more from the folly of its friends 
than from the onslaughts of its enemies. Calculations 
are not ends in themselves, but amateurs sometimes 
delight in making them so, much as a small boy de- 
lights in whittling with his first pocket knife. Each 
calculation that is made should be made for a definite 
purpose, clearly seen beforehand. The science of sta- 
tistics is like any ordinary machine or tool. Its oper- 
ator must assume responsibility for the planning and 
direction of its work. A physician does not mechanic- 
ally add the patient’s temperature, pulse rate, and 
blood pressure and divide by three to find the nature 
of the malady, but novices in educational research do 
comparable things with data. When the statistician 
becomes a part of his machine, without the ability or 
inclination to plan and reason about his work, he should 
stop figuring. The average statistical study involves 
more errors of policy, aim, or underlying philosophy 
than of computation or mathematical performance. 
The latter can'be eliminated by the use of instruments 
and careful checking, but the former are not so easily 
detected. Someone has suggested that the statistician 
should beware of letting his mathematics smother his 
common sense. That is, he should check his work for 
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its logical soundness as well as for its technical accur- 
acy. Formulae and figures are good as far as they go, 
but they do not go the whole way. To borrow a figure 
from city life, it is well to ride the street car to the 
end of the line, but after that it is necessary to get off 
and walk the rest of the way. It is a wise man who 
knows enough to stop figuring when figures have ex- 
hausted their possibilities, or to refrain from beginning 
to figure until he knows what the figuring will do for 
him. : 

3. How to learn statistics. The work of the statisti- 
clan is a highly specialized one, calling for definite 
training in regular classes or schools. When more re- 
search workers come to appreciate this fact the qual- 
ity of the output will be improved. Since, however, 
there are numbers of persons who are already in situa- 
tions which call for statistical knowledge but who lack 
that systematic preparation, the following points may 
be of some service as guides of a self-help nature: 
(1) The beginner should study an extremely simplified 
book rather than one written for the training of real 
statisticians. In fact, it is sometimes better to find a 
statistical chapter in a book on tests and measurements 
or child study than to try to study a regular statistical 
book which goes more deeply into the technical aspects 
of the subject. Such methods do not make a statisti- 
cian, but one must learn to crawl before he can run or 
dance. (2) It is well to compare treatments of a given 
process, as finding the standard deviation, in two or 
more books, since each explanation is in different 
words and involves different illustrations. (3) Reading 
journal articles which involve statistical techniques 
adds a kind of understanding not furnished by ordi- 
nary textbooks, since the processes are carried out in 
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their natural settings or in connection with definite 
practical problems, whereas textbook explanations pre- 
sent the problems in the abstract. If the reading of 
the articles is accompanied by reference to the text- 
books on statistics, the results are greatly increased. 
(4) If a problem is once begun, the need for statistical 
tools will furnish such a powerful motive for their 
acquisition that ordinary statistical skills may be ac- 
quired to satisfy the immediate needs. (5) For many 
purposes learning how to use formulas mechanically 
without understanding their deriviation or theoretical 
background is all that is necessary. Many able persons 
can compute coefficients of correlation, and can inter- 
pret them, but cannot write the formula from memory 
nor tell why it is arranged as it is. 

4. Accuracy. Every statistical calculation should be 
checked for accuracy, else one error may do more harm 
than the rest of the work can atone for. Work should 
be considered wrong and conclusions false until the 
opposite is proved. The following suggestions should 
be of some service in the promotion of accuracy in this 
work: (1) Every table or column of data should be 
clearly labeled at the time it is written down. The 
results of elaborate experiments have been known to be 
reversed simply because of accidental mistaking of the 
scores for one group for those of the other. (2) The 
use of slide rules, adding machines, tables of squares, 
reciprocals, etc., is to be encouraged. (3) Doing the 
same thing by two methods permits each method to 
check the other. For example, adding figures down- 
ward checks on adding them upward. (4) Simply look- 
ing for results that are “out of line” is a good way to 
detect errors. If an item is markedly different from 
others in its class, it may be right, but it should be 
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given special scrutiny instead of being taken for 
granted. (5) Common sense or estimation methods 
will reveal many errors that are hard to discover by 
more exact forms of checking. For example, to check 
the computed standard deviation by noting whether it 
includes about two-thirds of all the cases is often bet- 
ter than following through the computation in detail. 
(6) The original data and enough of the calculation 
should be included in the record or report to enable the 
checking for accuracy to be done over again later either 
by the investigator or by those who are to make use of 
his findings. 

5. Aids to calculation. There are several devices 
available for lightening the burden of statistical calcu- 
lation which should be brought to the attention at this 
time. They are as follows: 

(1) The slide rule. This is a small calculating instru- 
ment which costs from one to ten dollars according to 
quality, and which is especially good for multiplication, 
division, finding squares, cubes, square roots, cube 
roots, etc. It does not add or subtract, nor does it per- 
mit accuracy in calculations beyond the fourth digit, 
but most problems do not require greater accuracy 
than this. 

(2) Calculating machines. There are now available 
many calculating machines which may be had for the 
price of a typewriter, which will do almost any kind of 
computation desired. They are usually slower than the 
slide rule, but have the advantage of doing some things 
which the slide rule will not do, and also of doing the 
work with greater accuracy. 

(3) Logarithms. Tables of logarithms are useful in 
almost exactly the same situation and for the same pur- 
poses as the slide rule. They have the same limitation 
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as the slide rule, also, as regards accuracy, since they 
only yield four, or in some cases five, significant digits 
of the number. They are used constantly by reputable 
statisticians, however, in spite of these limitations. It 
should be stated here that the slide rule is an invention 
which grew out of logarithms, and is in a sense a 
graphic logarithm table. Logarithms require more 
time than the slide rule, and in the writer’s opinion are 
not nearly so useful to the statistician. 

(4) Tables. In recent years numbers of books of 
tables have been published to facilitate statistical work. 
They include tables of squares, square roots, recipro- 
cals, products, quotients, etc., for the different numbers 
up to 1000 or 10,000, as well as the Probable Errors, 
Standard Deviations etc., of different statistical quan- 
tities. Speed and accuracy are sometimes considerably 
increased by looking up what is desired in such a book 
of tables instead of calculating it independently. 


TABLE IV. 
AN ILLUSTRATION OF A FREQUENCY TABLE 
WITH INTERVALS OF ONE 


Age Frequency 
(Number of children) 


5 34 
4 6 
3 6 
2 2 
otal hia eee ies 17 


6. Selection of statistical tools. The difference be- 
tween the rough carpenter and the fine cabinet maker 
lies in the number and variety of tools which are used 
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skilfully. The man who knows only how to work with 
a saw and hammer is comparable to the statistician 
who knows only how to compute an average. A gsuc- 
cessful research statistician must not only possess a 
wide variety of statistical tools but also know how to 
choose them appropriately for their specialized func- 
tions. A finished job calls for the use of many differ- 
ent techniques. The remaining paragraphs of this 
chapter are devoted to brief descriptions of these vari- 
ous techniques. The purpose is to serve as an outline 
for further study rather than to complete the training 
required for statistical research. 


TABLE V. 


AN ILLUSTRATION OF A FREQUENCY TABLE 
WITH INTERVALS OF TEN 


Pac), Frequency 
140-9 A: 
130-9 ve 
120-9 6 
110-9 8 
100-9 10 
90-9 9 
80-9 7 
70-9 5 
60-9 2 
RODRIG forvsctart ten, are ts mite 50 


7. Frequency distribution. If we had a group of 
pre-school children whose ages were Osea Aeon 4 weds 
3, 2; 0, 4, 5, 4, 8, 2; 3, 5; 4, respectively , and wished 
to get a clear mental grasp of their ages we would make 
a “frequency distribution” by getting all of a kind to- 
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gether, as in Table IV. If, however, we were dealing 
with intelligence quotients of 50 second graders in 
which there was a range in scores from 60 to 145 we 
should find it convenient to bunch the scores by tens, as 
in Table V. Two slight variations of this are found 
when the frequencies are expressed in percentages of 
the total, and cumulatively, as in Table VI. These fre- 
quencies may also be represented graphically, giving a 
still more vivid picture of the distribution of the scores. 


TABLE VI. 


AN ILLUSTRATION OF FREQUENCY TABLES IN 
RAW FREQUENCIES, PERCENTAGES, AND 
PERCENTAGES CUMULATIVE. 


bho Luise fea ie f 


in% in% 
cumulative 
140-9 1 2 100 
1350-9 ta ak 98 
120-9 256512 94 
110-9 8 16 82 
100-9 10 20 66 
90-9 9 18 46 
80-9 7 14 28 
70-9 5 10 14 
60-95 ea yA 4 
Total¥.:.. 50.100 — 


8. Central tendency. There are three common ways 
of finding the average, or central tendency, of a set of 
figures, as follows. (1) The mode. This is found by 
making a frequency distribution and observing which 
score has the highest frequency. In other words, it is 
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TABLE VII. 


AN ILLUSTRATION OF THE SHORT METHOD FOR 
COMPUTING THE MEAN 


—16 
M= 105 + (=) xX 10 = 101.8 
50 


Note: 
, mean 
AM, assumed mean 
, algebraic sum of 
» number of cases 
Int., class interval 
d, deviation from assumed mean 


the score that occurs most often. It ig not used very 
extensively. (2) The median. This is the mid-most 
score, or the one above and below which there are 
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equal numbers of scores, regardless of how far above 
or below it they chance to go. It is used quite com- 
monly as a method of eliminating the effects of a few 
exceptionally low or high scores. It is found simply by 
counting up to the middle of a frequency distribution 
from either end and stopping when half of the total 
number of scores have been passed. (3) The mean. 
This is the ordinary average, determined by adding 
all the scores and dividing the total by the number of 
scores. Since, however, this is usually rather long, 
and since also the standard deviation is also desired 
at the same time as the average, a short method is 
commonly used which consists of computing deviations 
from an assumed or guessed mean and making a cor- 
rection for the error in guessing later. A sample of 
such a computation is given in Table VII, based on the 
frequency distribution previously used. This proced- 
ure should be studied more in detail in a regular statis- 
tics book, after which this brief presentation may be 
used for reference. 

9. Variability or dispersion. In addition to learning 
the central tendency, or average, it is usually import- 
ant to know how closely the scores concentrate around 
it and how widely they vary or deviate from it. There 
are several ways in which this can be determined, as 
follows: (1) Frequency curves. Simply drawing a graph 
to represent the frequencies shows in a visual way how 
much they vary from the center of the distribution. 
(2) Quartiles. These are determined in the same fashion 
as the median, except that, instead of finding the mid- 
dle point of the distribution, the points are located 
above which there are three-fourths and one-fourth, 
respectively, of all the cases. (3) Standard deviation. 
This measure is based both upon the number and the 


STATISTICAL TECHNIQUE 227 


amount of the deviations from the central tendency. 
In an approximate way, it is defined as that distance 
above and below the average which it would be neces- 
sary to go to include the middle two-thirds of the cases. 
It is easily computed as a byproduct of the calculations 
already shown to illustrate finding the mean. In order 
to illustrate it and to review the computation of the 
mean, another example is given in Table VIII, adding 
only one column especially for the standard deviation. 
Since the mean for this distribution is 18.73 and the 
S. D. 5.58, we are prepared to say that approximately 
two-thirds of all the scores, or 22 scores, would be 
included between the limits 13.73-++5.58, or 19.31, and 
13.73—5.58, or 8.15, a fact which can be verified by 
observing the distribution itself. (4) Probable error. 
This is derived from the standard deviation by simply 
multiplying the latter by the constant, .6745. It is 
interpreted in exactly the same way as the standard 
deviation, except that it is the distance above and be- 
low the mean required to include the middle fifty per 
cent of the cases instead of the middle two-thirds. 
Thus, in the example given, since S.D. is 5.58, P.E. is 
.67455.58, or 3.77. Consequently, we should find 
about one-half of the 33 cases between the limits 
13.73+3.77 and 13.73—3.77. 

10. Correlation. Two variables are Said to be cor- 
related if a variation in one ig accompanied by a defin- 
ite and systematic variation in the other. For example, 
Since weight of children increases as their height in- 
creases, they are said to be positively correlated. Like- 
wise since the number of errors made in practicing 
typewriting tends to decrease as the amount of prac- 
tice increases, we say that errors are negatively cor- 
related with amount of practice. The degree of cor- 
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TABLE VIII 


AN ILLUSTRATION OF THE COMPUTATION 
OF THE MEAN AND STANDARD 
DEVIATION TOGETHER 


Lear) 


Scores 


9 
M = 138 + — X 3 = 13.73 
33 


Std? /Sfd\ ? Sfd 
Formula: SD = (=) X Int. 


Sfd?=117 
N 


~ Vea} 


=5.58 


Therefore: 
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relation may vary all the way from perfect positive 
(1.00) through zero correlation (0.00) to perfect nega- 
tive (-1.00). When there is a certain amount of agree- 
ment but not perfect, it results in a fractional coeffi- 
cient, as .56, .74, -.82, etc. For practical purposes cor- 
relations below about .20 are ordinarily considered neg- 
ligible, .20-.40 low, .40-.60 substantial, .60-.80 high, 
and .80-1.00 very high. Care should be observed in 
regard to interpreting correlation coefficients, to avoid 
assuming that correlation proves causal relationship. 
For example, weight and height are positively corre- 
lated, but weight can hardly be said to cause the height, 
nor height the weight, since they are both dependent 
upon such other factors as age, hereditary influences, 
etc. The accompanying model for computing the co- 
efficient of correlation, shown in Table IX, is made very 
simple by including only a small range of scores, and 
the beginner will find it necessary to study the subject 
in regular statistics books, but after once learning the 
procedure, this model may be useful for reference. No 
attempt is made here to explain the derivation of the 
formula. The reader may take for granted that it is 
true, or else study its derivation elsewhere. All steps 
in the first four columns at the side and at the bottom 
of the table are already familiar to the reader who has 
studied the procedure for the mean and standard devia- 
tion, the letters x and y representing deviations of the 
respective variables from the assumed means. The 
numbers in small type within the various cells are 
simply the products of the individual cell frequencies 
and their respective x values. The numbers at the 
bottom of the table in the fx row are simply the alge- 
braic totals of the numbers in small type. The totals 
for the fx column should be the same as for the fx row, 


TABLE IX 


ILLUSTRATION OF THE PROCESS OF COMPUT- 
ING A COEFFICIENT OF CORRELATION. 


(Correlating arithmetic grades with test scores.) 


Arithmetic Grades 
Test Scores E D C B A 4: Xo] ee 


27-29 1B eb “tl 9? 21) 
206 | | | |5 dt yii2 > 4) esl 4 coe aee 
Shop) | ie he ['a) lia. ©, 4 9) sok wo)| Soe ae 
192001 1 8 ANS dia vt |[ ad 
ily [22d 8 id @ asl 2 A) isle | eee 
eta fi lL ToS |e ge 
f 7) 16: ) 29°]! 49 | veep 1a ea eae 
y ea aaa Rate bs 

Fy ie eat nee Tas ata be seat ae Roce 

fy? | 231 16! of 49| 52 | 197 

fixe ih 17 | tee (emote eee e Dien ad 

fey tL rea ea | RO Aah e9 

Formula: 


(Sfx) (Sfy) 
Riey ee aera 


(Sfx) 2 (Sfy) 2 
Sfx? — Sfy? — 
N N 


In this problem: 


Sfxy= 99 
Sfx = —42 
Sfy= 71 

Sfx2= 224 

Sfy2?= 197 

N= 127 


Substituting, we find 
r=.57 
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since they are both the same thing arrived at by dit- 
ferent routes. If these do not check, there is some 
error in the work. The fxy row is based on multiplying 
each fx by its corresponding y value three rows above 
it. It should be noted that correlations may be com- 
puted between scores in numerical form or between 
letter grades, the main essential being that the inter- 
vals used are equal steps throughout for a given 
variable. 

11. Curvilinear correlation. The procedure for the 
preceding paragraph is based on the assumption that 
the relationship is of a straight-line type, but this is 
not always the case. For example, we might presum- 
ably find a curved-line correlation between size of class 
and efficiency of instruction on the assumption that 
very small classes do poor work for lack of a social 
stimulus, that medium sized groups do better, and that 
extremely large ones break down from excess numbers. 
If such were the case, we would probably find no corre- 
lation by the straight-line formula but an appreciable 
amount of correlation if the other formula were ap- 
plied. The procedure of computing curvilinear correla- 
tion is too complicated to warrant presentation here. 
The research worker who finds a need for it should 
consult other statistical books for detailed directions. 
The method of finding whether the labor of computing 
it will be justified or not is to plot the scatter diagram 
as for the straight-line correlation and observe whether 
or not it shows a concentration along a curved line. 
Unless its curvilinearity is visible to the eye from the 
scatter diagram, it is probably not worth while to make 
the extra calculation. 

12. Partial correlation. It is not uncommon for two 
variables to be correlated with each other simply be- 
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cause each is dependent upon some third variable. For 
example, if we should find a correlation between a stu- 
dent’s notes and his grades, we would not know 
whether the notes affected the grades or whether good 
notes and good grades were both due to high intelli- 
gence. The device by which we can answer this ques- 
tion is called partial correlation. It seeks to find what 
the correlation between two variables would be if the 
effect of a third one were held constant. Of if a fourth, 
fifth, or still other factors are suspected of being 
present, they may also be held constant. Thus partial 
correlation serves somewhat the same purpose in 
statistical work as controlling conditions to hold all 
variables constant except the one to be studied plays 
in experimental work. . The experiment would not 
allow these other variables to enter into the situation. 
The statistical study lets them operate as they will, 
and then discounts their influence by means of partial 
correlation. The formula for holding one variable 
constant is: 


ri2—(¥13)(Te3) 
V1—(ri3)? Vi—(res)2 
The formula for holding two variables constant is 


E235 


D4 9.144 (ri 3.4)(T23. 4) 

TAR: 9 Ron eile ts ceeeraes teach eae tte nee ese tote 

T(r ae Vaan we 
13. Multiple correlation. The purposes of multiple 
correlation is to determine what correlation would re- 
sult if two or more scores were weighted in the most 
favorable proportions possible and this composite score 
correlated with some third score, or criterion. For 
example, the investigator wishes to predict success in 
algebra. He has already correlated intelligence with 
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algebra grades, getting a coefficient of say .54. He has 
correlated arithmetic and algebra grades, getting a co- 
efficient of say .68. The correlation between arithme- 
tic grades and intelligence he has found to be, say .37. 
His problem then is whether a weighting of the arith- 
metic grades and the intelligence scores in the most 
favorable manner possible would result in a better cor- 
relation than arithmetic alone gave. The formula for 
this is: 


r?y,e+r?,3—2ri2XrisXIri3 
Ri. 23 = ERPS SSE) OATS Re er oe ee Soe GET Tees 
1—r2o3 


Thus in our own case, calling algebra grades 1, intelli- 
gence scores 2, and arithmetic grades 3, we have 


(.54) 2+-(.68) 2-2 X .54X .68 X.37 
Ri.23 = eit CMs cue eae kt EP aEd We cas © A Geaeak cca ok at ae 
1 —(.37) 2 
= .87 


Since .87 is appreciably higher than .68, it is clear that 
it would be desirable to make the composite score by 
weighting the intelligence scores and arithmetic grades. 
Otis gives a formula* for determining the best weights, 
which is as follows: 

Total score = 1st score + W(2d score) 


where 
Tistrig$ Tes SD1 


Tie—Yi3s Yo SD2 


For formulas for multiple correlation with many 
variables, the reader should consult Buckingham’s 
article.** 

*Otis, Arthur S, Statistical Method in Educational Measure- 
ment, World Book Company, Yonkers, New York, 1925, p. 241. 


**Buckingham, B. R., Multiple Correlation, in Journal of 
Educational Research, Vol. IX, Feb., 1924, 149-154. 
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14, Regression equations. The purpose of the re- 
gression equation is to predict the most probable score 
in one variable from a knowledge of the score in 
another variable that is correlated with it. Thus, 
going back to our illustration of the correlation 
between arithmetic grades and test scores, we see that 
they are correlated to the extent of .57. If a child’s 
grade is E, what did he most likely make on the test, 
or supposing he missed the test, what would he prob- 
ably have made if he had taken it? The formula for 
predicting x from y is 


(Sfx) (Sfy) 
Sry 


(Sfx) 2 
Sfx? — 


It will be observed that these items are all available 
from the correlation computation sheet. Similar but 
somewhat more complicated formulas are available for 
predicting one variable from the combined knowledge 
of several other variables. 

15. Reliability. Since practically all statistical cal- 
culations are based on limited samplings taken from 
the total population, there is always a question or 
doubt as to whether the findings for the observed 
sampling would hold true for the whole group, or for 
other similar samplings taken from it. It is for the 
purpose of answering this question that reliability 
must be determined for any statistical quantity which 
is based on a sampling. When reliability has been 
determined, it is possible to know how certain the 
same results would be to occur again upon further 
sampling. For example, it is possible to predict from 


STATISTICAL TECHNIQUE 235 


a given observed sampling what the odds are that 
further samplings would give results above zero. It 
is impossible to explain here the theory of the formulas 
for reliability, except to say that they are based on 
the mathematics of probability and the normal prob- 
ability curve. 

We shall only state a common rule of the statistician 
that for a quantity to be “significant” or for us to 
possess “practical certainty” that it would stil] hold 
true for other samplings, it should be 4.0 times its 
Probable Error (P.E.) or 2.78 times its Standard 
Deviation, (S.D.) There are two common procedures 
for checking up a quantity to see whether it meets 
this test, one being the “critical ratio” and the other 
the “experimental coefficient.” The critical ratio is 
simply the ratio between the quantity and its P.E., 
expressed in decimals. For example, if the mean of 
of the sampling were 16.3 and its P.E. were 3.9, the 


critical ratio would be or 4.2. Since a ratio of 


4.0 indicates significance, or practical certainty, we 
would say that we are certain that other samplings 
would yield a mean somewhere above zero. The logic 
of the experimental coefficient* ig the Same as that of the 
critical ratio. Using the illustration above, with a 
mean of 16.3 and P.E. of 3.9, we would go back to 
the 8.D. which would be 5.8. The ratio between the 


16 
mean, 16.3 and its S.D., 5.8, is , or 2.82. Since 
5.78 


this ratio should be 2.78 or more in order to have 


*This device is described in great detail in McCall’s How to 
Experiment in Education, and seems to be of his own invention. 
The writer considers it one of the most convenient methods of 
testing reliability that are available. 
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practical certainty, it is clear that it slightly more 
than meets the test. But how much more? This 
may be answered by computing a further ratio, namely 


2.82 
, or 1.01. For convenience both of these steps 


2.78 
are combined into one formula, which is as follows: 
M 
E. C. (Experimental coefficient) = ——————- 
2.788.D.M. 


The same thing may be expressed in the correspond- 
ing form for Probable Error, which would be: 


M 
C. = 
4 P.K.M. 


It now remains to give formulas for determining 
reliability for various statistical quantities. These 
will be expressed in terms of S.D. and experimental 
coefficients, since P.E. and critical ratios may be easily 
derived from these if desired. The P.E. is always 
.6745 8.D., and for a quantity to be 4 times its P.E. 
is practically the same as for it to be 2.78 times its 
8.D. 


(1) Reliability of a mean: 


S.D. scores 
S.D.M. = —————- 


— 


VN 

M 
Cc, = ————_— 
2.78 S.D.M. 


(2) Reliability of a difference between two means: 
D — M t —M 2 
$.D.D. = VS.D.M.?2,+8.D.M.?.2 
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D 
a) = aaa 
2.78 S.D.D. 


(3) Reliability of a sum of two means: 
S. = M itM 2 
S.D.S. = /8.D.M. : it+sS.D.M. 2 2 


s 
2.78 S.D.S. 


(4) Reliability of a sum of three or more means: 


S.=M.i+M.+M;+ sree es +Mn 
§.D.S. =-/8S.D.M.?21+8.D.M.?2,+8.D.M.?2,+.. +S.D.M.?2n 


S 
eel 
2.78 S.D.S. 


(5) Reliability of a difference between the sums of two or more groups 
of means: 


D.=S:i-—S, 

8.D.D.=/SDS7i+5.DS2, 
D 

2.78 $.D.D. 


(6) Reliability of a standard deviation: 
S.D. 


S.D.sd = cae 
VN 
S.D. 


E.C, =——_—_—_—_ 
2.78,8.D.sd 
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(7) Reliability of a correlation coefficient: 


1—r? 
SD = a 
JN 
r 
Cc. =——__—_ 
PAE batter BY: 


(8) Reliability of a difference between two correlation coefficients: 
D.=ri-r2 
S.D.D. = /8.D.r?,+8.D.r?, 
D 


E.C.=————————— 
2.78 §.D.D. 


(9) Reliability of a partial correlation coefficient: 
1L—r712.3 

./N- 
T12.3 


E.C, =———_____ 
2.78 No WP ok 12-3 


S.D.r 12-3—> 


(10) Reliability of a multiple correlation coefficient: 
1—R?24.23 
S.D.R. 1-23 or ohare Vicente ie 
/N 
R 1-23 
2.78 S.D.R. 1.23 


= 


McCall gives the values of the E. C. up to 1.5 in 
terms of chances that further samplings would yield 
quantities above zero.* Thus an E. C. of .1 involves 

*McCall does not carry his table beyond 1.5, but it is readily 


seen that the odds are so enormous for an E. C., of this size or 
larger that there is no need of the exact numbers. 
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chances of 1.6 to 1; .2, 2.5 to 1; .8, 3.9 to 1; .4, 6.5 
Mie. 10.156, 20 tol: 7, 38: to.1;).8;:7b tol: 
7, L60 to 1; 1.0, 369 to 1; 1.1 930. to 1; 1.2, 2350 to 1: 
1.8, 6700 to 1; 1.4, 20,000 to 1; 1.5 65000 to 1. 


EXERCISES AND QUESTIONS 


1, Why is the statistical method considered so important in 
educational research? 


2. Criticize the statement that ability to use a formula is more 
important than understanding how it is derived. 


3. How can a greater amount of common sense be combined 
with statistical research? 


4, What are the main qualifications essential for a successful 
statistician? How can these best be acquired? 

5. What justification can you give for the statement that slide- 
rule accuracy is usually as much as is needed in educa- 
tional research? 


6. Formulate a number of research problems from your own 
schools that would involve the statistical technique. 

7. What are the dangers in presenting a brief and simple state- 
ment of statistical technique, such as the one in this 


chapter? What are the advantages? How may dangers 
be avoided? 
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CHAPTER XIV 
TABULAR AND GRAPHIC TECHNIQUE 


The value of graphs. An old proverb has it that 
“one picture is better than a thousand words.” Not 
infrequently graphs will convey an idea better than 
any amount of verbal explanation and better than 
the most elaborate tables of statistical data. They 
not only have a power to convince which has long 
been appreciated by the advertising experts, but they 
have another value beyond this. They are excellent 
devices to aid the research worker to analyze his facts 
and to study or understand what they signify.. This 
function of graphs is often not duly appreciated, since 
the common idea is that graphs are to show others 
what has been discovered, rather than to help the 
research worker to discover it. The making of a 
graph, therefore, is not necessarily a final step in the 
Study. It is more often a step leading to a more 
complete discovery of the Meaning and significance 
of the data. 

In passing, we may mention the fact that sometimes 
graphs prove to be useful tools or labor-saving devices 
in certain statistical operations. For example, the 
process of converting a large number of intelligence 
test scores into percentile ranks or of turning minutes 
into percents of .a class hour, may be facilitated by 
plotting the line to represent the relationship between 
the two variable and reading the results from the 
line instead of making a mathematica] calculation each 
time. 
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Since the use of graphical and tabular devices is 
so essential in research work, the remaining portion 
of this chapter is devoted to some ways to increase 
their effectiveness. 


I. Types of Graphs 


1. Bar graphs. A very common type of graphical 
representation of facts is the bar graph. It consists 
of a rectangular bar for each group, individual, city, 
or item that is being described. The lengths of the 
bars indicate the relative magnitude of the cases 
compared. The bars may represent such things as 
the total expenditures of different cities, the average 
scores of several classes on an arithmetic test, or the 
average amount of improvement in handwriting for 
each of several classes during a semester’s time. Like- 
wise, bars may be used to compare boys and girls as 
to a number of abilities, with a pair of bars to show 
each comparison, one pair showing average grades of 
boys and girls in arithmetic, another pair in reading, 
etc. 

2. Spot graphs. There are many kinds of problems 
which require a method of showing how the individual 
cases are distributed over a given geographical area, 
or over a given range of statistical scores. For this 
purpose a map may be drawn and upon it a spot may 
be marked for each case, or else a pin may be stuck 
in it for each case. Thus a spot or pin may be used to 
locate each consolidated school in a county, each high 
school student’s residence in a city, or the spot at 
which each juvenile crime was committed. 

A slightly different kind of spot graph is that in 
which a set of scores is arranged into class intervals 
in which one dot is marked beside each interval to 
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represent one case, the frequencies being shown by 
the number of dots or tallies recorded. 

The scatter diagram ig a slightly more complicated 
form of the spot graph just mentioned. It involves 
locating spots so that each shall represent the status 
of a given case with respect to two different variables. 
For example, height may be shown along the hori- 
zontal axis, and weight along the vertical, and a dot 
located at that point which would express both the 
correct height and the correct weight for the child. 

3. Organization charts. A type of graph which is 
exceedingly useful in studying the administrative 
organization of a school is the organization chart. It 
involves the use of a square or rectangle for each 
official or member of the school staff, with the con- 
necting lines to show sources of authority and respon- 
sibility for each. Imperfections in the organization 
which are very difficult to discover otherwise become 
glaring realities when reduced to such graphic form. 

4, Time series charts. In statistical] investigations 
which trace a given phenomenon through a series of 
months or years, it is frequently desirable to draw a 
line to show the course of the phenomenon, with its 
various ups and downs. The familiar charts to be 
Seen in the newspapers showing the trends of bond 
prices, living costs, etc., by years or months illustrate 
this type of chart. Similar charts for school data 
are common, of which the learning curve is probably 
the most familiar. 

d. Circle graphs. Not infrequently a graph is 
needed to show amounts of the different parts com- 
posing the whole of something, as, for example, the 
different items composing the total expenses of the 
school, or the amounts of the different sciences included 
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in a course in general science. A very effective way 
to show such differences is by means of the circle 
graph, with each component part represented by a 
“pie slice.’ This is usually preferable to the use of 
concentric circles, and also better than squares within 
squares, since it leaves as the sole basis of comparison 
between the parts the portion of the circumference 
which represents them. When concentric circles are 
used for the various parts of the whole, the compar- 
isons are based on length of radius instead of length 
of circumference, and are therefore deceiving, because 
areas of pie slices are directly proportional to the 
circumference but areas of circles vary as the square 
of the radius. 

6. Histograms and frequency polygons. One of the 
most common situations calling for graphical repre- 
sentation is that in which it is desired to show the 
actual distribution of all the cases in a given group, 
or to compare two groups or sets of scores upon the 
basis of all cases, rather than simply on the basis of 
averages. For this purpose, the histogram and fre- 
quency polygon are very useful. They simply involve 
graphic presentation of the facts shown by the fre- 
quency distribution (See chapter on statistical tech- 
nique), either in the rectangular form that results 
from allowing a square to stand for each case, or else 
in the polygon form that results from joining the 
midpoints to represent frequencies. 

A problem in the use of frequency polygons which 
sometimes gives trouble is that of comparing groups of 
different sizes. This is facilitated if the polygons 
are reduced to the same scale. The easiest way to 
do this is ordinarily by expressing the frequencies in 
the form of percent of the whole. Thus, if there are 
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thirty students in one group and forty in another, 
the graphs for the two groups may be reduced to the 
same scale quite readily by reducing them both to 
the scale of one hundred. 

7. Percentile graphs. The percentile graph is a 
slight but very useful modification of the frequency 
polygon. It is based on the ordinary frequency dis- 
tribution but involves two interesting variations. (1) 
The frequencies are cumulative, each being added to 
the total below it, so that the frequency recorded at 
the end of the range is the total of all frequencies in 
the whole distribution, or one hundred percent of the 
cases. (2) These cumulative frequencies are next 
converted into percents of the total frequencies, and 
these percents are used in the actual construction of 
the graph. Needless to say, two or more percentile 
graphs may be drawn upon the same paper, thus com- 
paring scores for different groups, schools, etc., in 
much the same way as is done by the use of frequency 
polygons. Very full and detailed treatment of this 
type of graphs is to be found in Otis’ Statistical Method 


in Educational Measurement. 


II. Captions for Tables and Graphs 


Because of the common tendency of beginners in 
research to neglect certain essentials regarding the 
titles for tables and graphs, and labels for their var- 
ious parts, it has been thought best to include here 
a few directions concerning these points. 

1. Numbering. <A standard rule for tabular and 
graphic work is that each table and each graph should 
bear a definite serial number. It is customary to 
number tables and graphs in separate series, using 
Roman numbers for tables and Arabic numbers for 
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graphs. The practice of presenting numerical mater- 
ials informally, and without the dignity of a definite 
table number, is especially to be condemned, as is also 
that of presenting some charts or diagrams as “strays” 
or “orphans” while others are recognized as important 
enough to receive serial numbers. If tabular or 
graphic materials are important enough to be included 
in the report, they should be important enough to 
receive recognition in the number series. 

2. Title for the table or graph as a whole. Each 
table or graph should bear not only a serial number, 
but also a title or caption which is expressive of its 
meaning or contents. It is unwise to make it neces- 
sary that such explanatory matter be secured from 
the body of the text. The caption should tell clearly 
whatever is needed for the understanding of the table 
or graph, or, if doing so would make it too long or 
voluminous, it should tell it in a brief way and refer 
to an explanatory note at the bottom of the page, in 
which may be included all necessary detail. Special 
care should be exercised to avoid errors of capitaliza- 
tion in titles. A common error is that of attempting 
to capitalize the first letters of principal words, a 
method which is seldom done in any consistent way. 
Ordinarily it is better to capitalize only proper names. 
It is customary to place the number and title for a 
table above the table and for a graph below the chart. 

3. Identification of the parts. Not only should each 
table or chart as a whole be identified clearly, but also 
each of its separate parts. To assume that the reader 
will understand what each line or set of numbers stands 
for ig to invite error or confusion. If there are dif- 
ferent colors, or solid and broken lines, a “legend” 
may be shown, indicating what each color or each kind 
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of line stands for. The same purpose may be served 
by arrows pointing to the different items, leading in 
from explanatory labels written in uncrowded portions 
of the page, and also by titles written parallel to or 
above the parts to be identified. A good rule is to 
leave no column, row of figures, line, or area without 
a clear identification of some kind to enable it to 
be easily understood. 


ITI. Suggestions Regarding Form 


There are many matters of form and arrangement 
of tables and graphs which are of quite general appli- 
cation. It is not possible to present here a complete 
list of rules and standards for good form. Such lists 
may be found in the books referred to at the close of 
this chapter. The suggestions presented here are 
those which have grown out of the examination of 
about fifty research reports, chiefly master’s theses, 
and cover the points which seemed to be in greatest 
need of improvement. 

1. Suitability to the purpose. It frequently happens 
that the same data may be presented in many different 
ways, or by several different kinds of tables and 
graphs. In such cases one should not be satisfied 
until he has drawn up each of the several types to 
compare their effectiveness. A little experimenting 
with different plans with a pencil and rough paper 
will ordinarily go far toward discovering the exact 
type of graph or table that is best for the particular 
purpose that is to be served. 

2. Loose-leaf plan. Each table or graph should be 
put on a page by itself in the manuscript, instead of 
sharing a page with another table or graph or with 
the subject matter of the written report. This is 
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because the typist or printer cannot adjust the report 
to make such units come at any certain predetermined 
portion of the page, and will be obliged to insert them 
at other places than that of the writer. If they are 
on separate pages, they allow maximum freedom of 
arrangement by the typist or printer. 


3. Boundary lines. As a general rule a table or 
graph should be surrounded by lines to show its limits. 
Particularly does this apply to a graph, since the 
numbers which accompany it must be arranged at its 
edge, and are more effective if they are along a definite 
line. Not every graph or table requires boundary 
lines, but many times they would be a real improve- 
ment. 


4. Neatness. Careless work in tables and graphs 
is distinctly undesirable. Since the whole purpose is 
to present a visual appeal, neatness is extremely 
essential. One of the greatest aids to neatness is 
the use of rulers and instruments to secure straight- 
ness and smoothness of lines. Lines drawn freehand 
are usually slightly irregular and present an untidy 
appearance. The use of small circles or dots to locate 
intersection points is also superior to the use of crosses, 
as the latter present an untidy appearance. Evenness 
in the heaviness or darkness of lines also is an 
essential of neatness. 


5. Simplicity. It is unwise to attempt to show 
everything in a single table or graph. As a rule each 
one should be a unit, or present a single message. 
The principal justification for showing two things in 
the same table or graph is in order to accomplish the 
single result of comparing the two. When a table or 
graph is made to show several different things, it 
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frequently fails to show any of them effectively be- 
cause of their obscuring each other. Also, the more 
complex it becomes, the less certain it is of being 
understood. The safest thing is to present each fact 
or idea in a table or graph of its own, and then to 
present comparisons or summaries in separate ones 
which have the sole purpose of comparing or of 
summarizing. 

6. Secale. The scale to which a graph is drawn has 
much to do with its effectiveness, or even its truthful- 
ness. Facts can be distorted by the simple device of 
magnifying the vertical scale and reducing the hori- 
zontal scale so that the two are out of proportion. 
Petty differences can be magnified in this way until 
they appear to be very important differences. After 
having determined the proportions which will set forth 
the facts truly, it is often important to experiment 
further to select the size or scale which will best fit 
the paper used or that will harmonize best in appear- 
ance with what is adjacent in the report. 

7. Smoothing. The effectiveness of a graph is often 
spoiled because of extreme variations up and down so 
that no general trend is readily observable. Two learn- 
ing curves on the same page, for example, may cross 
and criss-cross so many times that it is difficult to say 
which is the better. Such a difficulty is easily solved 
by means of smoothing, or drawing the general trend 
of the data without the details. Several methods of 
smoothing may be used, but one very simple one is 
that of reducing the number of class intervals by in- 
creasing their size. A learning curve which presents 
an entry for each three-day average will be smoother 
than one which gives a separate entry for each day. 
Another way to smooth a curve is to reduce the scale 
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of the graph in the direction in which the fluctuations 
occur. In addition to these devices, there are the ordi- 
nary statistical procedures for smoothing. 

8. Orderly arrangement of items. When several 
items are presented in a table or graph, they should be 
arranged according to some systematic plan and not 
be entered in miscellaneous order. Thus the states of 
the Union may be listed alphabetically, in order of rank 
in population, or on some similar basis. Bar diagrams 
might well be arranged in ascending order of rank as 
to length of the bars, and “pie slices” in a circle graph 
should be arranged so that thick and thin slices are not 
alternated. 

9. Grouping for comparison. When graphs or tables 
are to be used for purposes of comparing two things, 
the items to be compared should be so grouped as to 
lend themselves to such comparisons. Thus, adjacent 
columns in the tables should be assigned to the things 
to be compared instead of allowing intervening col- 
umnsg to be occupied by figures relating to some other 
aspect of the situation. Thus if each of two groups 
is described in terms of “number” and ‘“‘per cent,” there 
should be a pair of columns for “number” and a pair 
for “‘per cent,” instead of a pair for “Group I’ and a 
pair for “Group II.”’ Similarly if bar graphs are to be 
used for comparison, it is important that the bars be 
grouped in pairs close together, with wider spaces be- 
tween pairs than between the members of a pair. 

10. Zero point shown. To omit the zero point in a 
graph is often enough to defeat the whole purpose of 
the graph. For example, if two bars are used to con- 
trast average grades of boys and girls, the boys aver- 
aging 84.89 and the girls 84.25, to begin the bars with 
84.00 as the base line would leave the impression that 
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the girls had grades only about a third as high as the 
boys when in reality the grades are almost identical. 
If including the zero point would make the graph too 
long, a break may be made between it and the end, 
shown by a wavy line, to indicate that a part of the 
intervening space has been left out. 

11. Numbers to accompany graphs. The numerical 
data upon which a graph is based should always be 
made available. As a rule, it is well to put this infor- 
mation on the page with the graph itself by writing 
the numbers along the boundaries of the graph or by 
presenting them in tabular form beside or below the 
graph. Care should be exercised to avoid locating num- 
bers at the ends of bars in such a way as to give the 
visual impression of lengthening the bars. When 
graphs are wide, it may be well to record the number 
scale at both sides, or at both the top and the bottom, 
to facilitate reading quantities from either edge. In 
some cases this plan may be varied by putting the 
actual numbers on one side and their equivalents in per 
cent on the other. 

12. Reading position. A table or graph should be 
so constructed as to permit it to be read from the bot- 
tom of the page or from the right. For this reason, all 
writing should be either horizontal or upward, but 
never downward, since writing downward necessitates 
turning the page so the table is viewed from the left. 

13. Relation of tables and graphs to the text. A good 
rule in the preparation of tables and graphs is to 
assume that the reader has not and will not read the 
discussion in the text, and that his sole information 
about the things treated in the tables and graphs comes 
from them only. If possible, they should be clear 
enough to be intelligible without the text. Then the 
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discussion in the text might well be written very much 
as if the tables and graphs did not exist, with special 
care taken to make the discussion intelligible without 
referring to the tables and graphs for interpretation. 
This does not mean, however, that the text should 
enumerate all that is presented in tabular form, but 
rather that what it does treat should be intelligible in 
itself. Each table or graph presented on a page to it- 
self as mentioned before, will necessarily have to be 
referred to in the text. This should not be done in 
such a way as “The following table shows ” but 
“Table XVI shows »’ since it will be impossible to 
predict whether in the final report the table will fol- 
low or precede the reference to it. 


IV. Materials and Procedures in Constructiny 
Tables and Graphs 


It is not within the scope of this chapter to enter into 
detailed discussion of drawing materials or technique. 
At best we can hope here only to point out in a general 
way a few of the principal forms of equipment and a 
few suggestions on technique which are applicable to 
the person who has not had special training in mechani- 
cal drawing. . 

1. Typewriting. By exercising a little forethought 
it is possible to prepare many tables and graphs so that 
they can be copied on the typewriter and thus be made 
a standard rather than a technical process. In order 
to do this, it is necessary to be familiar with the type- 
writer keyboard and mechanism, and to consider with 
some care the number of spaces included in a line of 
typewriting, as well as the space required to typewrite 
particular words. Some graphs are such that all but a 
few lines can be produced on the typewriter, and these 
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can be inked in afterward. As a general rule, whatever 
can be done on the typewriter can also be done in print, 
since the printer has a greater freedom in selecting and 
arranging type than has the typist. It is not always 
economical, however, to have a table or graph set up 
by the typist or printer, as the labor charge may be 
greater than that involved in other processes. 

2. Lettering. Writing the numbers, titles, and iden- 
tifications of the various parts of tables or graphs pre- 
sents difficulty to persons who have not had special 
training in the art of freehand lettering. There are 
several ways of overcoming this difficulty, as follows: 
(1) It is possible to hire this work done by persons 
who have had the necessary training. (2) The letter- 
ing may be done on the typewriter, special care being 
taken to get a ribbon which writes with the same de- 
gree of darkness as the other parts of the graph. (3) 
It is possible to purchase transparent pieces of celluloid 
substance with the various letters and numbers cut 
through to serve as a guide to the pencil or pen. 
These, however, are usually difficult to operate. (4) 
For some purposes ordinary handwriting, if of good 
quality, may be entirely adequate. As a general rule, 
good handwriting is preferable to poor lettering. 

3. Pens. Special drawing pens of various types are 
available for those who can use them. For the ama- 
teur, however, it may be better to work with more 
familiar tools. One very simple device of non-technical 
type, however, will be of considerable service, partic- 
ularly in the making of wide lines or bars. It is 
possible to secure pens with points as wide as a fourth 
of an inch, which will make a smooth and even line 
with one stroke. The commercial name of one such 
set of pens is “Speed Ball Pens.” Such pens overcome 
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the difficulty of having a bar made up of light and 
dark streaks due to uneven distribution of the ink 
on different parts of the paper. 

4. India Ink. The use of India Ink in graphs is 
important for a number of reasons. It is opaque, and 
serves to stop the light from going through the paper, 
thus lending itself well to the preparation of blue- 
prints. It is superior, also, for photographing pur- 
poses, and is, therefore, essential in any graph that 
is to be reproduced by photostat or engraving. It 
makes a very sharp and distinct impression which lends 
effectiveness to a graph even though it is not to be 
reproduced at all. 

There are a few points which the amateur should 
know about using India Ink: (1) It forms a cake 
when it dries and hence should be cleaned out of pens 
and instruments promptly. (2) It should be allowed 
to dry on the paper, and never be blotted as other 
ink is, since blotting removes practically the whole 
impression. (3) It should never be erased in the usual 
way. Erasures, if made at all, should be made by a 
sharp knife or razor blade, removing the cake of ink 
and not disturbing the tissue of the paper. (4) 
Corrections should be made before rather than after 
erasures are made, the new letter or number being 
written over the old, and then the old erased after 
the whole is thoroughly dry. If an erasure is once 
made, the new and moist ink tends to spread through 
the roughened paper, making it impossible to get 
a neat line. 

5. Colors and their substitutes. Not infrequently it 
is desired to represent different items by different 
colors, as, for example, the different nationalities in 
a school, or the errors and the time in a learning curve. 
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If the graphs are to be made entirely by hand and 
never reproduced in quantities, this is quite feasible, 
and the contrasting colors are very effective. But if 
they are to be copied, colors are quite a problem, as 
color printing is very expensive. 

The common substitute for colors is the device of 
varying the texture of the line while using plain black. 
Thus one line may be solid and another dotted, one 
bar solid and another hollow or cross-hatched in dif- 
ferent directions, etc. Graphs constructed on this 
principle may be photographed or blueprinted without 
difficulty. 

6. Cross section paper. It is possible to secure paper 
ruled according to almost any imaginable pattern and 
suitable for almost any kind of tabular or graphical 
work. It is well for the research worker to have at 
hand samples of many different patterns so that he 
can choose the style or scale that fits his needs best. 
It is not necessary, in every case, for the cross lines 
to be shown on the graph itself, but such lines are 
necessary as guides in constructing the graphs. A 
helpful device in such a case is to draw the graph on 
transparent paper with a sheet of cross section paper 
underneath to furnish a guide to go by. If lines are 
needed in the graph to help read it, it is usually pre- 
ferable to use the printed graph paper instead of 
drawing cross lines especially for the purpose, both 
for reasons of economy and of increased effectiveness 
of the graph. Cross section paper can be bought or 
ordered through any good bookstore or stationer. 

7. Blueprinting. One of the cheapest and most satis- 
factory ways of duplicating a table or graph is by 
blueprinting it. The first essential is that the material 
to be copied be drawn on transparent paper or tracing 
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cloth. Ordinary onion skin, or even thin typewriter 
paper, will serve the purpose. This copy should be 
done in India Ink, or else on the typewriter. If done 
on the typewriter, a very black ribbon should be used, 
and a black or red carbon paper placed underneath in 
such a position as to make a second copy of the 
material on the back of the paper. These two layers 
of black materials ordinarily serve about as well as 
one layer of India Ink. An ordinary black lead pencil 
and a carbon paper may also be used in much the 
same way as the typewriter. 

When the copy has been prepared, by any one of 
these methods, it is sent to a blueprint company, who 
make copies of it exactly as if it were a photograph 
negative. The cost of such copies is very low indeed, 
usually only a few cents a square foot. 

If a positive print is desired, a second copy, or Van 
Dyke print, may be made from the original, this being 
just the reverse of the original tracing, and from it 
may be printed any desired number of blueprints which 
are dark where the tracing is dark and light where it 
is light. 

8. Photostatic processes. A very useful device for 
reproducing tables, graphs, or similar materials is the 
photostat process. This involves simply the making 
of a photograph of a page or chart, reducing or 
enlarging it to such size as desired. This makes it 
possible to draw a very large graph and then reduce 
it to a standard page size, in which minor errors and 
inaccuracies of the drawing would be reduced so 
greatly as to disappear from notice. This device 
permits of making a clear and readable repro- 
duction that is smaller than the hand could possibly 
make. In preparing work to be photostated, it is 
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necessary to be careful lest certain lines be too faint 
to show at all on the reduced copy. 

Closely related to the photostat process is engraving. 
It involves simply taking the picture on a sensitized 
metal plate, after which the metal is dissolved out 
chemically except at those places where the light has 
caused a chemical which is insoluble to be formed. 
These parts which are not dissolved thus stand up 
higher and receive the printer’s ink and print black 
whereas the others do not, thus forming a printed 
copy of the original graph. 

9. Folding. Not infrequently a chart or table is 
larger than the standard page on which the research 
report is to appear, thus being in a sense in the 
“awkward squad.” Two remedies for this are possible. 
The chart may be reduced in size by the photostat 
process, or it may be folded and inserted. If the latter 
is done, care should be taken to be sure that when 
the report is bound, the folded material can still be 
opened up for inspection, and also that the outer folded 
edge does not come near enough to the right to permit 
the trimming knife to cut it. In other words, it should 
be a bit undersized when folded. As a general rule 
it is well to avoid graphs and tables which require 
much folding, and to use instead a larger number of 
graphs which present parts of the findings separately. 


EXERCISES AND QUESTIONS 


1. How does the value of graphs to the research worker com- 
pare with their value to the readers of the research 
report? 


2. Arrange the various types of graphs mentioned in this chap- 
ter according to their order of usefulness in educational 
research. Justify your rating. 
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8. Using data supplied in the chapter on statistical technique, 
construct for yourself as many different types of graphs 
as you can. 

4, Let each member of the class construct a graph to repre- 
sent the same set of data and then compare the different 
graphs made. 


5. Read some article or book that has many graphs and criti- 
cize the effectiveness of the various graphs used. 


Go through a published research report and note as many 
things in it as you can that could be representd graphi- 
cally. 
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LIBRARY TECHNIQUE 


1. Research based on books. There are several fac- 
tors which operate in favor of basing research activi- 
ties upon the study of books instead of upon experi- 
ments, tests, interviews, and similar direct sources of 
information: (1) The investigator may be so situated 
that the latter types of procedures are not available 
to him, as in the case of full-time graduate students 
who do not have schools or classes of their own to 
study directly, and who therefore find it more con- 
venient to resort to books and libraries. (2) Barriers 
of space and of time sometimes make it difficult to 
study a particular problem except through the medium 
of printed materials which bear on it. (8) Some prob- 
lems, and some fields of knowledge, lend themselves 
very little to other than bookish procedures, as, for 
example, legal problems or problems in literature and 
history. (4) Some research problems have to do with 
the comparison or classification of viewpoints which 
leading writers have expressed about certain questions, 
in which case their publications naturally constitute 
the chief source of information available. 

In view of these factors, and of the common ten- 
dency of thesis writers and others beginning research 
to rely too heavily upon printed materials instead of 
upon assembling unpublished data, a warning should 
be issued against indiscriminate and unscientific use 
of books in research. The fact that a study is to be 
based on books rather than on concrete observations 
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does not excuse the research worker from his respon- 
sibility as a scientist. He should treat his printed 
materials as systematically and as critically and as 
objectively as he would any other kind. Furthermore, 
he should formulate his problem just as carefully and 
precisely as if he were employing some other kind of 
technique. 

The various situations in which reading enters into 
the life of the research workers, and the principal 
points in the technique of using books scientifically 
are treated in the remaining paragraphs of this 
chapter. 

2. Reading about research technique. One type of 
reading which the prospective research worker must 
necessarily do is that designed to acquaint him with 
the technique he is to employ in his investigations. He 
must read to train himself in scientific metnods of 
work. If he proposes to do historical research, he 
should by all means read books on historical method. 
Likewise one who expects to do experimental, statis- 
tical, or job analysis research should learn all he can 
about these procedures by such reading as he can do. 
If he can get access to regular courses in research 
methods, so much the better, but, if not, he must make 
up the lack by his own self-directed effort. This may 
be accomplished in part in connection with graduate 
courses, seminars, and term papers, but much of it 
will probably remain to be done as a part of the more 
immediate preparation for the actual writing of a 
thesis or getting started off on an investigation. Un- 
less considerable training in research technique has 
already been received, probably the best thing a begin- 
ning research worker could do would be to devote a 
few weeks to a campaign of self-education in the 
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techniques he expects to employ in his investigation. 

3. Reading fer general background. Another type 
of reading which should be done by a research worker 
is that which provides an orientation in the general 
field in which his investigation is to lie. For example, 
an education student must be familiar with the history, 
psychology, sociology, philosophy, administration, 
methods, measurement, and other phases of education 
before he is competent to do really scientific work in 
any one of these lines. If he has not had such a back- 
ground or orientation, the reading of at least one 
outstanding book in each of these lines should be done 
as a corrective for the lack. Also it would be well, as 
a means of providing a more immediate form of back- 
ground, to read all articles in a few current issues of 
the leading educational journals, whether they are 
related to the thesis problem or not. Such a reading 
of a representative sampling of recent educational 
writings provides somewhat a cross-section view of 
the field of education which broadens the view and 
improves the perspective. 

In addition to these types of orientation, it may 
sometimes be necessary to do a different type of 
reading by going over into the related subjects. An 
education student will profit by reading books on 
sociology, biology, anthropology, etc., in addition to 
his background reading in the field of education itself. 

4. Reading about the particular research problem. 
Next after reading on research technique and reading 
for general background, comes reading on the specific 
problem which is to be investigated. This has several 
aspects, or factors, to be considered: (1) Merely 
familiarizing oneself with a number of writings on the 
immediate problem that is to be studied gives a rich- 
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ness of mental association about it which serves as 
excellent material in thinking about it. (2) The con- 
clusions reached by other investigators serve some- 
what as proved theorems in geometry upon which 
later work may be based without going back to prove 
them anew each time they are needed. (8) Such 
reading helps to suggest new problems that are to be 
solved. (4) It enables one to avoid doing again what 
other investigators have previously done. (5) It 
enables credit to be given where due to previous in- 
vestigators. (6) It suggests improved techniques that 
might not be discovered by working alone. (7) It 
warns against faulty or inadequate techniques that 
have already been tried out and found wanting. (8) 
It enables one to revise his own research plans before 
he does his experiment instead of after he has spent 
his time and energy upon its performance. (9) It 
safeguards his originality if he will read many or all 
other studies, since the reading of only one or two will 
leave him a slave to them, whereas a large number 
will present so many viewpoints as to produce a state 
of mental independence. (10) Extensive reading of 
similar studies after one’s own study has been finished 
aids in interpreting the results which have been 
secured. (11) If the other studies which are read are 
in the form both of articles in journals and also in 
complete theses, they supplement each other admir- 
ably, since the articles yield a clear perspective because 
of their condensed nature, and the full reports give 
a wealth of detail on the technique of the investiga- 
tions. (12) The results of the reading of other inves- 
tigations on the problem may be incorporated into the 
research report in the form of a chapter with such 
a title as “Survey of Related Investigations,” and thus 
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give the reader the same sort of orientation as that 
which the research worker gained by doing the reading. 

5. Using books as original sources. It is entirely 
possible to conduct investigations based upon published 
materials and have as high standards of scientific 
merit as if the data used were secured from original 
research. For example, the United States Census 
Reports contain large quantities of data which may 
be used in the solution of problems which the govern- 
mental authorities have not attacked. Likewise, many 
research reports which have been published contain 
data which are capable of being treated in other ways 
than have been attempted by their authors. A new 
investigator may take such a report, with its tables 
of original data, and ‘‘work up” this material to solve 
other problems than those which have been solved, 
or to check and verify the solutions arrived at by 
others. 

There is still another sense in which books and 
printed matter may be used as original sources of 
information. An illustration is involved in such a 
problem as “A survey of Junior High School English 
as revealed by printed city courses of study,” which 
would call for an examination and tabulation of the 
contents of the city courses of study from a number 
of different cities. Such materials are just as truly 
original data as are scores on tests or replies to ques- 
tionnaires. In the effort to be objective and scientific, 
it is well to keep a sense of perspective, and to beware 
of rejecting or neglecting sources of information 
merely because they are in print. Not everything that 
is in print may be called original data, by any means, 
but much of it may be used in scientific ways if utilized 
properly. 
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6. Using books as secondary sources. Many times 
a book cannot be truly considered as an original source, 
but may nevertheless be extremely useful as an aid 
in a given piece of research. A secondary source, or 
a book which talks about some original data collected 
by others rather than one which presents original data 
of its own, may be helpful in locating the original data, 
and also in interpreting them. It may be used to 
stimulate a check-back to the original data, and a 
verification of them; or it may give a clearer idea 
about what facts are lacking, and thus direct the search 
for new data of an original sort. Furthermore, the fact 
that a given book or article is simply a discussion of 
the actual facts presented by some other author does 
not by any means indicate that it lacks sicentific merit. 
If a good scientist makes a critical estimate of another 
person’s investigation, his report is a secondary source 
but a quite reliable and valuable one. It is better to 
depend upon good secondary sources than upon poor 
and inadequate original ones. 

7. “Interviewing” books. <A type of investigation 
which may well occupy the time of graduate students 
and research workers is the taking stock of educational 
writings upon a set of definite questions, issues, or 
problems. For instance, the topic for the investigation 
might be “A survey of the status of theme-writing in 
the published works on the teaching of English.” This 
would then be particularized by the formulation of 
such questions as: (1) Is the recommendation favor- 
able or opposed to theme-writing? (2) How long ~ 
should themes be? (8) How should topics be selected? 
(4) How should themes be criticised? etc. The inves- 
tigator would then examine carefully the published 
materials on the teaching of English to find how each 
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author answered each of these questions, and thus 
secure data upon the way that body of experts reacts 
to theme writing. Such a procedure would permit of 
numerical tabulations regarding the number who fav- 
ored assigning topics as contrasted with the number 
who favored freedom of choice by the students, and 
similar questions. A slightly different style of “inter- 
viewing” that might be applied to books would be that 
which seeks to discover and list in one place the 
methods suggested by the various writers for perform- 
ing some particular operation or solving some teaching 
difficulty. Thus the investigator might go through a 
number of books for their answers to such questions 
as “How to make Latin contribute the maximum 
amount to the improvement of English.” The results 
of interviewing books might be handled in such ways 
as are suggested in the chapter which treats interview 
technique. Such a plan is ordinarily better than 
simply reading at large upon a subject and writing 
a paper on what seems to stand out in the mind as 
a result of the contacts made. 

8. Making comparisons and contrasts. A type of 
problem which permits of a scholarly and systematic 
use of printed materials is that which makes compar- 
isons or contrasts between two bodies of literature. 
For example “A comparison of the philosophies of 
Plato and John Dewey” affords a chance for a rather 
far-reaching comparison, based upon a search through 
two rather large bodies of writings for parallels and 
contrasts upon a number of different points. Another 
type of comparison and contrast is that illustrated by 
the Literary Digest, which presents the whole range of 
views, opinions, and attitudes that have been expressed 
about a given question. While such procedures are 
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not suited to statistical treatment, as a general rule, 
and therefore may not appeal to persons who hold 
strongly to the ideal of extreme objectivity in research, 
they may have much to commend them especially if 
they are used by competent persons and with due 
regard for their limitations. 

9. Analyzing space distribution in books. A form 
of investigation which is particularly suited to use 
with printed materials is the study of the amount of 
space given by different books to particular types of 
subject matter. For example, general science text- 
books may be studied with respect to the percentage 
of space given respectively to physics, chemistry, 
botany, zoology, physiography, etc. Or arithmetic 
textbooks may be examined as to the amount of space 
given to drill exercises as contrasted with presenta- 
tion of principles, or to reasoning problems as con- 
trasted with pure drill problems, etc. This procedure 
is applicable also to materials other than textbooks. 
The books on the teaching of art, for example, may be 
analyzed to determine relative space given to art appre- 
ciation, design, painting, etc. Such studies may be 
made by counting the number of chapters, pages, lines, 
or even words devoted to the different types of 
material, the size of unit depending upon the nature 
of the problem and of the material used. 

10. Preserving objectivity in using published mate- 
rials. A caution which is especially in order in connec- 
tion with the use of secondary printed materials is 
that against subjective treatment of these materials. 
An opinion stated by an author may be accepted as 
his opinion, and treated as such, without sacrificing 
standards, but the research worker should make it 
very clear that the opinion is so recognized and used. 


LIBRARY TECHNIQUE 269 


The fact that an opinion is held by even a foremost 
authority is not necessarily enough to establish a fact. 
Even that person may have been guessing or mistaken. 
It is well to look for the evidence which formed the 
foundation for the opinion, and if none is available, 
to put the responsibility for a given statement upon 
the person who made it instead of taking personal 
responsibility for its truth. It is better to express 
other person’s opinions as theirs rather than to express 
them as if they were one’s own, even if the latter is 
accompanied by a footnote reference to the previous 
author. In other words, it is well to shift the respon- 
sibility for statements of opinion upon those who have 
expressed them, just as it is well to put the responsi- 
bility for strictly scientific conclusions upon the orig- 
inal data instead of bearing that responsibility oneself. 
Even if a viewpoint seems unimpeachable, one’s own 
attempt to defend it before others may not be so suc- 
cessful. 

11. Finding references. The technique of making a 
bibliography is a rather specialized one, and one which 
calls for some little special training. Ordinarily the 
graduate student or research worker will have had 
access to some such training in his previous educational 
experience. If this is not the case, he might do some 
special reading and study on the art of using a library, 
learning the special functions of the various guides 
and the special techniques involved in using them. 
Some of the leading indexes which should be learned 
are the Card Catalog, United States Catalog, Cumulative 
Book Index, Reader’s Guide, Reader's Guide Supplement, 
International Index, Poole’s Index to Periodical Literature, 
and various encyclopedias, yearbooks, and _ special 
bibliographies. 
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EXERCISES AND QUESTIONS 


1. Explain why research is so commonly associated with books 
and libraries in the popular mind. 


2. Compare the use of books and libraries with the use of 
other techniques discussed in this book as to their scien- 
tific value. 


3. Is a “reading thesis” more or less. likely to be scientific 
than an experimental one? Why? 


4. Formulate a number of research problems that seem to lend 
themselves particularly well to a reading type of re- 
search. 


5. Read a few reports of research based on reading and criti- 
cize the procedures and conclusions reached. 

6. How would research reading differ from reading for other 
purposes? Why? 

7. What types of books or other library materials should the 


library purchase in order to make the greatest possible 
contribution to educational research? 
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CHAPTER XVI 
ANALYSIS AND INTERPRETATION OF DATA 


Importance of analysis and interpretation. A clerk 
can do the routine portion of an investigation, but it 
takes a real research worker to interpret the results 
and draw conclusions after the clerical work is finished. 
Mere juggling of figures or tabulating facts is not re- 
search. Research involves a considerable amount of 
reflective thinking. Drawing conclusions is a thought- 
ful rather than a mechanical process, and cannot be 
done in a mechanical way. Nor can it be done hastily. It 
requires a maturity of consideration which necessitates 
a long period of time while it is in process. Research 
workers have been known to find facts with infinite care 
and precision and then to explain them with the utmost 
recklessness, giving no reasons for their conclusions 
or giving as reasons for them what could be at best 
mere guessing. Certainly the finishing stages of an 
investigation deserve to be done with as great care as 
the other parts; yet they are frequently done with the 
least care of all. 

Two reasons explain a large amount of the neglect 
of the analysis and interpretation phase of investiga- 
tions. (1) The effort to be objective sometimes comes 
to be a fetish in the mind of the one making the study, 
so that he cannot, even for a moment, let himself resort 
to anything which in the least way resembles the resort 
to opinion or speculation. And yet drawing conclu- 
sions must necessarily consist in the exercise of 
rational or speculative processes in connection with the 
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facts which have been assembled. (2) The drawing of 
conclusions is not uncommonly left to the last, at which 
time the investigator has become thoroughly tired of 
his study, or has developed a condition of mental stale- 
ness which causes him to shirk the strenuous process 
of critical analysis and interpretation. More effort to 
formulate tentative conclusions as the study progresses, 
and to record these for later consideration, would serve 
to some extent as a corrective. 

There are two principal evils to be oor era found 
in research reports. One is the evil of failing to get 
out of the data all the conclusions of which they are 
capable; the other is the evil of drawing conclusions 
not warranted by the data. The remaining paragraphs 
in this chapter will offer a few suggestions to help rem- 
edy these evils. 


I. Making the Data Yield All Conclusions 
of Which They Are Capable 


In the effort to discover everything of significance 
which may be secured from the data, many different 
procedures and devices may be employed. The follow- 
ing are a few of these: 

1. Reading. After the data have been assembled 
and such conclusions drawn as seem justified, a period 
of reading on the subject may aid in extending the 
process of thinking the matter over, and thus help to 
produce other significant conclusions. This reading 
may be directed toward other investigations which have 
been made in the same field, or toward the writings of 
persons who have dealt in a less scientific way with 
matters related to the problem. In fact, the reading 
of unscientific materials may direct attention to issues 
or phases of the question which would be overlooked 
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if the reading were limited to the strictly research ma- 
terials that have been published. 

2. Consulting the records of the study. If proper 
records have been kept as the study progressed, they 
should be fruitful sources of mental stimulation when 
the time comes for drawing conclusions. These records 
should contain not only the actual data collected, but 
also information about the numerous conditioning fac- 
tors and circumstances that accompanied the collec- 
tion of the data. Since the conclusions must necessar- 
ily take account of these conditioning factors, a careful 
survey and minute examination of all records of the in- 
vestigation should add something to the list of conclu- 
sions to be drawn. 

3. Subdividing the whole. It is ordinarily better to 
begin the search for conclusions by examining each part 
or aspect separately rather than by attempting to in- 
terpret the investigation as a single unit. Failure to 
draw adequate conclusions is simply due in many cases . 
to working with too large a unit. Not infrequently, a 
part may suggest as many significant conclusions as 
does the whole, thus enriching the results as many 
times as there are parts so analyzed. Simply dividing 
the study up into its phases, aspects, or chapters is, 
therefore, a fruitful way of studying it for significant 
conclusions. 

4, Placing items in new combinations. It is some- 
times possible to discover new significance in a fact 
simply by placing it alongside another fact. It is said 
that the invention of the Eskimo Pie was due to the 
accidental dropping of some chocolate frosting into an 
ice cream freezer, upon which it promptly hardened 
and thus revealed a new principle which has resulted 
in thousands of dollars worth of business to the con- 
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fectioners. Proximity is a factor favorable to discov- 
ery of likeness, cause and effect, etc. Hence the mere 
putting of facts into positions adjacent to each other 
may aid in revealing their hidden meaning. Or, con- 
versely, if two things happen together, they may profit- 
ably be examined to see if they are related to each 
other in any way; or if one follows the other, it may be 
examined to see if it was caused by it. Frequently the 
failure to draw sufficient conclusions is due to the fail- 
ure to find the connecting links between facts which are 
together and related, or to put them togetney so that 
their relation could be observed. 

5. Classifying. To classify facts involves getting 
all of a kind together and separating all that are differ- 
ent from each other. This enables a generalization to 
be made regarding each class or group of items, and 
thus promotes reducing the whole to a state of order, 
system, or law. When all cases of a kind are together, 
their agreement or uniformity is much more apparent, 
and much more accurate limits or boundaries can be 
set for the laws which are involved in them. If the 
same facts are classified, cross-classified, and reclassi- 
fied on many different bases or from many different 
viewpoints, their full significance is more likely to be 
discovered than if they are only classified in one way. 
Using only one classification means getting only a very 
small fraction of the potential meaning of the facts. 

6. Generalization. The chief aim of most research 
is to discover the uniformities, laws, principles, or gen- 
eralizations that run through a large number of spe- 
cific cases. Until a fact or number of facts can be 
reduced to such a law or formula, the problem has not 
been solved, nor the work of the investigator finished. 
In other words, the conclusions reached should, in a 
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sense, represent the central tendency, or average, or 
most truly representative feature of all the individual 
instances which have been studied. This requires 
classification, comparison, and contrast, and leads to a 
discovery of the common elements running through 
the various situations or cases, with the definite recog- 
nition of the fact that these are the common elements. 

7. Looking for exceptions. A single instance which 
differs in some way from all of its fellows is a poten- 
tial source of knowledge about some new factor not 
previously known. New planets have been discovered 
by the astronomers because of their further study of 
planets which seemed to disobey the known laws of 
astronomy as if being acted upon by other forces. Any 
fact or case that seems to disagree with a hypothesis 
or generally accepted belief, if properly followed up, 
may lead to a revision or correction of that hypothesis. 
Each exception is a signal for further thought to see 
why it is different. 

8. Treating the data statistically. The failure to 
draw sufficient conclusions from a study is sometimes 
due to failure to apply the available statistical tech- 
niques to them. If the mean or median is computed 
and the process stopped at that point, naturally the 
conclusions are meager. But if in addition to a single 
measure of central tendency there is computed other 
central tendencies, measures of dispersion, correlation, 
alienation, reliability, etc., the number of conclusions 
that may be drawn is greatly increased. The research 
worker should make an extra effort to apply to his 
data every procedure or form of treatment which is 
appropriate to them and which can be used in a pur- 
poseful way instead of stopping when he has applied 
one method of treatment. 
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9. Writing a report of the investigation. Until an 
investigation has been written up in full and complete 
form, the full meaning of it is not grasped by the in- 
vestigator. The act of writing up the results refines 
and completes the thinking and interpretation of the 
facts assembled. Writing thoughts down defines and 
clarifies them, and brings into prominence ideas which 
previously were present only in the fringe of conscious- 
ness. Any defects or errors in the conclusions reached 
are more likely to be discovered if recorded in actual 
black and white. In short, the surest way to get all 
the conclusions out of a study which it is capable of 
yielding is to report the study in full and complete 
form. An investigation which is carried out “just for 
curiosity,” and is never written up in definite form for 
a thesis or for publication as an article, is almost cer- 
tain to be a relatively unfinished job, and to stop before 
it reaches many of its potential conclusions. 


IIT. Guarding Against Unwarranted Conclusions 


The previous section has dealt chiefly with the prob- 
lem of discovering the significance of data and formu- 
lating all possible conclusions; the present section is 
to consider the ways by which the drawing of conclu- 
sions not justiged by the facts can be avoided. 

1. Verifying major premises. Just as a building 
placed upon a weak foundation cannot stand, so a long 
chain of reasoning or an elaborate structure of theory 
built upon a false major premise is certain to topple 
sooner or later. For this reason, it is extremely import- 
ant to avoid building upon untested assumptions. It 
is unsafe to take anything for granted, as it may turn 
out to be false and invalidate all work done from that 
point on. 
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2. Checking for internal consistency. One check 
that may be employed to test the value and soundness 
of conclusions drawn is whether or not they are con- 
sistent with themselves. If the various parts of an in- 
tellectual structure or theory that has been built up 
fail to agree with each other, the probability is that 
something is in error. For all aspects of an elaborate 
body of findings to be consistent with each other does 
not establish beyond a doubt that they are all true, but 
it does furnish rather strong circumstantial evidence 
that such is the case. 

3. Checking against external criteria. Not only 
should all conclusions be consistent with each other, 
but also with objective facts or external realities. Ex- 
ternal consistency is a safer test of the soundness of 
theories than internal consistency. However, consist- 
ent a line of reasoning may be, it should not be accepted 
as true until it has been verified in the light of other 
criteria of some kind. Unless the theories actually 
work out in practice, they are of doubtful worth. It is 
safer to put faith in the senses than in reason when 
the two disagree, and to adjust the reasoning to what 
the senses reveal rather than to shut out from the 
senses what fails to agree with the conclusions pre- 
viously reached. | 

Incidentally, we might remark that a very useful ex- 
ternal criterion by which to check conclusions drawn 
from an investigation is the body of facts and conclu- 
sions arrived at by the other investigators who have 
worked in the field. If the findings of a study are not 
in agreement with those of other studies that have 
been made, they may well be submitted to a careful re- 
examination for possible errors. 

4, Getting the criticisms of other people. It is well, 
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in a certain sense, to be a pessimist about conclusions 
that are drawn, and to assume that they are all wrong 
until they are proven right. Or, to put it differently, it 
is well to assume that no one will accept the conclusions 
which have been drawn, and that all who read them 
will find fault with them. This consciousness of an 
audience to be convinced and satisfied is a very whole- 
some and restraining influence, causing the investigator 
to examine his facts with much greater care than he 
would if he had only himself to satisfy. 

Not only should conclusions be drawn with a pros- 
pective audience in mind, but they should be checked 
by being actually submitted to such a critical audience. 
A presentation of the findings to a small group of re- 
search people, or to a graduate seminar, for criticism 
will usually result in a discovery of the weaknesses 
and fallacies that are involved, and thus give an oppor- 
tunity to make necessary revisions before publication. 
Such a practice, incidentally, would prevent the serving 
up of some of the “‘half-baked” conclusions which are 
put before the public each year and which do more to 
injure the cause of science than to advance it. 

5. Avoiding common fallacies. There are certain 
typical errors which are quite common not only in re- 
search but also in everyday life, which may well be 
mentioned here with the hope that if the investigator is 
forewarned, he may be prepared. They are as follows: 
(1) Mistaking correlation for causation. It is not 
necessarily true that two things are related to each 
other because they happen to be together. One might 
be the cause or the result of the other, or both might 
be caused by some third and unseen influence. Like- 
wise, it is unsafe to assume that because one event 
follows another it was caused by it. (2) Generalizing 
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upon too few cases. “One swallow does not make a 
summer,” but one instance of a phenomenon of human 
behavior is frequently the only basis for a geenraliza- 
tion about “human nature” whether it was a typical 
ease of behavior or not. This sort to error is extremely 
common. (3) Generalizing from unrepresentative data. 
Great care is needed to avoid assuming that all teach- 
ers believe the same as the few teachers who volun- 
teered to answer a given questionnaire, or that all 
children react in the same way as the group which was 
included in a given experimental study. The research 
worker is constantly tempted to state as general or 
universal laws the conclusions which hold true for the 
small group of cases included in his study. (4) Mis- 
taking a part for the whole. It is extremely difficult 
to avoid a tendency toward finality due to limited per- 
spective. The beggars, in Tolstoy’s story, who quar- 
relled with each other as to the nature of the elephants 
because each had felt a different portion of their bodies, 
are comparable to many students and scientists who 
disagree among themselves because each has discovered 
a different part of the truth and has mistaken it for 
the whole. (5) Comparing things that are not com- 
parable. This tendency to reason that because two 
things are similar in a few particulars, they are similar 
in all, is a most fruitful source of error. Such uncriti- 
eal reasoning by analogy has been one of the chief 
sources of superstitions and scientific errors. (6) 
Ignoring negative evidence. A common trait of human 
nature is the tendency to ignore cases which disagree 
with notions which are already held, treating them as 
“exceptions that prove the rule.” By thus shutting the 
eyes to opposing facts it is possible to build up a feel- 
ing of utmost certainty about matters that are abso- 
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lutely false. (7) Yielding to suggestion. The tendency 
to accept a statement as a fact without asking for a 
reason is a frequent source of error. This tendency 
is especially hard to overcome if the source of the 
assertion or suggestion is a person of considerable 
prestige or prominence. 

6. The law of parsimony. An old and very helpful 
rule for the scientist is that when several possible 
explanations of a phenomenon are available, the sim- 
plest should be chosen. “When in doubt, take the 
simpler interpretation,” and ‘‘Accept the simplest ex- 
planation that will account for all the facts,” are two 
common statements of this old “law of parsimony.” 
Another statement of the same general idea is ‘“That 
explanation is best which explains all the facts with 
the smallest number of assumptions.” A few examples 
will make clear the significance and importance of this 
rule. It is preferable to account for such a phenomenon 
as lightning by means of natural laws of physics than 
to attribute it to a special supernatural cause. Or, 
again, it is more in keeping with science to explain the 
nest-building of birds on the basis of simple instinct 
than to make the assumpton of intelligence and fore- 
sight as causes of the nest-building. In short, it is 
always preferable to accept the lower and simpler in 
preference to the higher and complex explanation, pro- 
vided it is equally able to account for all the facts. 
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EXERCISES AND QUESTIONS 


1. How does Dewey’s How We Think relate to the present 
chapter? 


2. Of what value would a course in logic be as a part of the 
training of a research worker? Why? 


3. Which do you consider the more serious error, failure to 
draw sufficient conclusions or drawing conclusions not 
warranted by the data? Why? 


4, Examine the findings and conclusions of a number of re- 
search reports and criticize the soundness and adequacy 
of the conclusions drawn. What types of shortcomings 
do you note? 


5. What are the surest guarantees of good results in drawing 
conclusions? 

6. What is the difference between the drawing of conclusions 
in science and in philosophy? 


7. Compare the value of internal and external consistency as 
standards for evaluating conclusions. 
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CHAPTER XVII 
REPORTING RESEARCH 


We shall consider the research report from two view- 
points. The first is that of what to include; the second, 
the form in which to present it. 


I. Content 


The types of material discussed in this section, while 
not essential to every thesis, constitute the most com- 
mon elements that deserve consideration in making the 
report. Incidentally, having such a list to go by, or 
such an outline to follow, may aid some struggling be- 
ginnergs to get into presentable form the materials 
which otherwise might be in part overlooked or else 
put into the records in a confused way. 

As a general rule, it should be possible, although it 
is not essential, to organize the report in such a way as 
to give each of these parts a separate status apart from 
the others, either as a chapter or as a separate division 
within a chapter or in some way which gives it definite 
treatment as such. Some of these divisions may be 
consolidated into one chapter when they are short; 
some of them, when long and involved, will require sev- 
eral chapters. 

1. Title. The title of a research report should be 
truly expressive of its contents. The use of short and 
catchy titles instead of significant and expressive ones 
is to be condemned, since the research report is of a 
more dignified nature and does not compete with the 
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journalistic type of writing. As a rule, the title is 
likely to be long, not because there is any virtue in a 
long title as such, but because the problem frequently 
does not permit of adequate statement in the briefer 
form. 

2. Tables of contents. Most research reports will 
require three tables of contents: one for the ordinary 
chapter divisions and subdivisions, one for tables, and 
one for graphs and charts. These will ordinarily be 
listed as “Table of Contents,” ‘‘List of Tables,” and 
“List of Graphs and Charts” respectively. The table 
of contents should be rather detailed and analytical, 
since it must serve also the purpose of an index for 
the unpublished report; and the lists of tables and of 
graphs should be arranged in serial fashion with the 
titles of the tables or graphs given. It is ordinarily 
undesirable to combine the list of tables with the list of 
graphs, unless the number of each is very small. 

3. Acknowledgements. A small space may be used, 
if desired, to acknowledge special courtesies and help 
rendered by persons outside of the line of duty, but 
ordinarily it is unnecessary to include in this section 
glowing tributes to the research advisor, since he has 
simply done what his position as research advisor pre- 
supposes. 

4, Problem. The opening chapter of the report 
should ordinarily be a statement of the problem investi- 
gated. -This should be so clearly stated that there can 
be no doubt or uncertainty as to what was sought in 
the study. Any vagueness in the presentation of the 
problem is likely to lead to vagueness or uncertainty 
in the later portions of the report. The problem may 
be stated in the form of a major question, followed by 
sub-questions, the answers to which are to be found in 
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the later chapters of the study. It is entirely permis- 
sible to state along with these the motives or reasons 
for the interest in them, or the benefits which might 
be expected to result from finding their answers. The 
problem chapter should not, however, constitute a 
“catch-all” for any miscellaneous material which is dear 
to the writer’s heart but does not fit in anywhere else. 
The use of the word “Problem” instead of “Introduc- 
tion” as the chapter title is preferable, since the latter 
tempts the writer to the practice just mentioned. 

5. Related investigations. The treatment of other 
investigations which have been made upon the problem 
may take either or both of two forms. It may occupy a 
special chapter, designated by title as a summary of 
related investigations, or it may be distributed through 
the body of the report with a mention here and a foot- 
note there as need arises. Ordinarily, a formal sum- 
mary in one place is more effective in showing the his- 
tory and present status of the problem, and in paving 
the way for the report on the present investigation. 
The giving of credit to other investigators in the body 
of the report should not be slighted, neither should 
footnotes and citations in the text be resorted to in a 
pedantic effort to appear scholarly. Such references, 
properly made, may add to the effectiveness of one’s 
expression, and may serve as corroboration for con- 
clusions arrived at, but when they are used just for 
show, they interfere with the reading of the report. 

6. Procedure. The chapter on the procedure of the 
investigation should not only tell in detail what ma- 
terials were used, what was done, and how the data 
were treated, but give also the reasons for doing these 
things as they were done. It should present the pro- 
cedure in sufficient detail to enable the reader to criti- 


290 THE TECHNIQUE OF RESEARCH 


cize and to judge its soundness, and also help him to 
make such judgments. The common tendency of the be- 
ginner is to make the procedure chapter entirely too 
short, or to mention a few of the high points of the pro- 
cedure in some obscure part of another chapter, as if it 
were a matter of small importance, when as a matter of 
fact it is of the utmost importance, and the foundation 
upon which the rest of the report must rest. The pro- 
cedure chapter should not only describe the procedure 
but also criticize it and present such evidence as is 
available to show how reliable or justifiable it was. It 
should show samples of the various questionnaires, 
directions, tests, or other materials used, and give any 
other information that is necessary to enable the 
reader to repeat the same identical procedure in a 
check experiment or investigation. 

7. Results. One or more chapters may be required 
to present the results. Under this head comes the 
tables, graphs, and non-statistical facts discovered, to- 
gether with the outcomes of special forms of treat- 
ment applied to them. The report should include not 
only the original data, but also the secondary data 
derived from them. Thus, if two sets of scores 
have been correlated, the report should include both 
the original scores and also the correlation coefficient 
derived from them. Omission of the former makes 
it impossible for any later investigator to check the 
latter. The results chapters should contain more than 
the bare statements of facts in tables, graphs, ete. 
These should be cemented together by running text 
material, which points out the significant or striking 
things shown in the tables and thus directs the reader’s 
attention while he surveys the results. 

8. Shortcomings and mistakes. Some weaknesses 
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are almost certain to be discovered in the procedure of 
every investigation by the time it is finished, however 
careful the investigator. Human nature is incapable 
of seeing far enough ahead to forestall every difficulty 
that might arise. Consequently, when it comes time 
to report the results, there will usually be a few short- 
comings or weaknesses which mar the picture. It is 
much better to point these out than to cover them up 
_ or ignore them. Nothing spoils confidence in a report 
so much as discovery of a few signs of concealment 
of its defects, and nothing helps to develop respect for 
a report quite so much as frank self-criticism by its 
author. If the reader’s doubts and questions are fore- 
stalled by the writer, or if the arguments on the oppo- 
site side of the question are presented before he has a 
chance to state them, he is much more likely to be con- 
vinced of the soundness of the conclusions reached. 
Furthermore, a statement of the pitfalls which were 
discovered in the study may aid later investigators to 
avoid them. 

9. Further work to be done. A striking characteris- 
tic of almost every investigation is its tendency to 
raise other related problems in the mind of the investi- 
gator. By the time a research worker has solved one 
problem, he can usually see dozens of others that are 
demanding solution. It would be unfortunate indeed 
to let these fruits of the study go to waste. A special 
section or chapter might well be devoted to “Further 
work to be done on this and allied problems,” in 
which might be mentioned new aspects of the problem 
that should be attacked, or new techniques which 
might be applied in the further study of aspects pre- 
viously investigated. After working with certain tools 
for a time, the investigator may discover ways of im- 


292 THE 'TECHNIQUE OF RESEARCH 


proving them or better ways of handling them, and if 
so these discoveries should be passed along for the 
benefit of others. 

10. By-products. It is a rare case when an investi- 
gation fails to reveal anything of value besides what it 
set out to reveal. Oftentimes the by-products are more 
important that the original problem. These should be 
saved, whether of major importance or not, simply as a 
matter of record and for the service of other investi- 
gators who may be interested in them. Two ways are 
open for embodying them in the report. One is to in- 
clude them in footnotes at such places as they seem to 
belong, thus relating them to the items which give rise 
to them without drawing attention in an undue degree 
away from the main stream of thought. The other 
plan is to devote a section or chapter to “By-Products” 
as such, and to present there all such findings as seem 
worth recording. If such a chapter seems to interfere 
too much with the main stream of the report, it may be 
placed in the appendix. 

11. Summary. At the end of a long and involved 
treatment of a topic, it is well to give a brief terse 
statement of its essentials simply to provide a “handle” 
by which it may be more effectively grasped, or as a 
crystallization of what has been suspended for some 
time in a state of diffusion, as it were. Some scientific 
journals make a practice of publishing a summary of 
each article at the beginning of the article, others at 
the close. The full research report, however, will ordi- 
narily require not only one general summary, but prob- 
ably also a number of smaller summaries of the various 
parts along in the body of the report. 

12. Conclusions. The chapter on “Conclusions” 
should constitute a natural counterpart to the chapter 
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on “Problem,” since it presents the final results of the 
attempt to answer the questions stated in the problem 
chapter. This chapter may be either a brief list of 
points serving as answers to the problem questions, or 
it may be a longer chapter made up of the answers 
accompanied by the analysis of the factors which en- 
tered into the final judgment on each. In either case, 
there should be somewhere in the chapter a few terse 
statements of the findings which might be copied by 
the reader to preserve the whole thing in a nutshell. In 
other words, there is as much need for summarizing 
the conclusions as for summarizing any other portion 
of the report, if not more need. 

13. Applications. Frequently the reader, on com- 
pleting his perusal of a research report, is inclined to 
say something like: “A very careful and thorough 
study, but what of it?” In other words, it is to him 
pure science, and has no practical applications. If he 
has an opportunity to ask the author of the report this 
question, he usually finds that the author can more 
than justify the study on the grounds of practical 
applications. This suggests the value of devoting a 
definite section of the report to applications in which 
the author may point out to his readers the practical 
significance of his findings, and thereby remove his 
study from the ranks of pure science. It would be 
most unusual if the author of a research report saw no 
practical application for his findings—as rare, in fact, 
as if the readers were able to see as many applications 
as the author could—and, this being the case, he should 
put his ideas on record. 7 

14. Appendix. There is considerable uninteresting 
material in every research report which interferes seri- 
ously with the reading of the report if embodied in 
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the text itself, but which should not be omitted entirely 
from the permanent records. The appendix is the logi- 
cal place for this. It may include the names of the 
schools or pupils studied, the lists of dates upon which 
observations were made, the technical procedure in- 
volved in statistical calculations, the bibliography 
which accompanies the investigation, or any one of a 
hundred other possible things. Sometimes the cost of 
printing a report may be materially reduced by setting 
the main essentials in ordinary type and putting every- 
thing else in small type in the appendix. | 


IT. Form 


Having considered what to put into the research re- 
port, we are now ready to consider the form in which 
to record it. The following suggestions may not apply 
to all cases, and may even be contradictory, in some 
instances, to the published rules and directions regard- 
ing theses for certain graduate schools. Without at- 
tempting to be dogmatic or final, we shall set them 
forth here in the hope that they may be helpful where 
no better guide is available. The reader is urged to 
secure copies of such official booklets of directions or 
form specifications as his school makes available, and © 
to follow them explicitly, relying upon the present 
chapter only in points which are not specified by such 
official publications. 

1. Displaying the outline. One of the most import- 
ant points regarding the form of the research report 
is that of displaying the outline. Instead of chapters 
that run from paragraph to paragraph without a break, 
the report should be organized into heads, subheads, 
and sometimes still smaller subheads, each with some 
form of prominent display. This plan of boldly setting 
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forth the main divisions and subdivisions enables the 
reader to skip and skim, and to find what he wants to 
know, as well as to get more meaning from each divi- 
sion. This display of the outline consists of two essen- 
tials, providing bold or prominent headings for the 
paragraphs and sections, and numbering or lettering 
these to indicate their relations to the outline. 

2. Announcing series of topics. It is well ordinarily 
to avoid abruptly beginning upon a series of parallel or 
coordinate topics without a brief introductory an- 
nouncement to prepare the mind for the series. Thus 
such a statement as “There were six steps in the pro- 
cedure, which will be outlined in detail in the following 
pages,” is preferable to simply beginning the descrip- 
tion of the first step. Also, it is well to make such 
transition statements, as “Having seen , we shall 
now turn to——-” in order to smooth the path from one 
unit to the next. 

3. Typographical details. To specify typographical 
details for a printed report is quite beyond the scope 
of the present paragraph, but there are a few hints 
which may be given here regarding the form for type- 
written reports, chief of which are the following: (1) 
The same style of type should be used for items of a 
given rank throughout the book, and should not vary 
from chapter to chapter. (2) The few major divisions 
of a chapter may be represented by headings placed in 
the middle of the page in a “balanced” position. 
These headings may be assigned Roman numerals, and 
may be written in all capitals. They should not be 
underlined, but they should be given prominence by 
spacing an extra line before and after, thus permitting 
the heading to stand out in the middle of a relatively 
open place. (3) Subdivisions of the second rank may 
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be numbered in Arabics, if at all, and begun at regu- 
lar paragraph position. Only the first word and proper 
names should be capitalized, and the entire heading 
should be underlined for prominence. The material of 
the paragraph should begin immediately at the end of 
the heading, wherever that may be, instead of at 
the beginning of a new line. (4) Subdivisions of still 
smaller rank may be assigned small letters instead of 
numerals and written the same as the next higher 
rank, without the underlining. (5) The plan of “cap- 
italizing main words” should be avoided, as it adds to 
the trouble and expense of typing, aids prominence 
very little, and is almost impossible to carry out con- 
sistently. When attempted, it usually results in cap- 
italizing some verbs but not all, or capitalizing long 
nouns and not short ones, and similar irregularities. 
The use of small letters or all capitals avoids these 
errors. (6) When long quotations are included, they 
should ordinarily be single spaced and indented slightly 
in order to be set off in appearance from the rest of 
the report. 

4. Starting new chapters and sections. Standard 
practice is to begin each new chapter at the top of a 
new page, whether the last chapter ended at the bot- 
tom of the page or not. This does not apply, however, 
to the subdivisions within a chapter, which should be- 
gin wherever the previous ones end. Nor is it good 
form to leave blank spaces or unfinished pages before 
tables or graphs. If the graph is announced or re- 
ferred to, or its discussion completed, in the middle of 
a page, the remainder of the page should be given to 
the discussion of the next one or the next topic in the 
chapter, avoiding the skipping of space. Otherwise a 
report that contains a large number of inserted pages 
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would present a very choppy and fragmentary appear- 
ance. 

5. Length of chapters. The length of a chapter 
should be determined by its subject matter rather than 
by arbitrary or external considerations. If, however, 
a given phase of the report is very long and involves 
a very complicated outline, convenience and clearness 
suggest that it be subdivided into several chapters in 
order to simplify the scheme of presenting the outline 
of heads and subheads. In other words, if several short 
chapters will present the message more effectively than 
one long one, they should be employed. 

6. Footnotes. The form for fotonotes can be readily 
observed by noticing almost any book, and need not 
take much space here. If there are many footnotes to 
the page, they should be referred to by means of num- 
bers, but if only one or two the use of the single or 
double asterisk is usually easier to employ. If the re- 
port is to be printed, the footnote should follow im- 
mediately after the line to which it refers, set off by 
lines before and after it, drawn across the page. If, 
however, the report is not to be printed, it is ordinarily 
preferable to place the footnote at the bottom of the 
page. 

7. Bibliographies. Standard forms for bibliograph- 
ies may be found by consulting the library card 
catalog, the Reader’s Guide, and similar library refer- 
ence works. It may be well here to mention the im- 
portance of recording complete information for each 
reference instead of just a few of the necessary items. 
To omit such things as publishers, editions, dates, etc., 
may make the reference entirely worthless. Biblio- 
graphies should ordinarily be arranged alphabetically 
by authors, and attempts to classify the works into 
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such divisions as books, monographs, and articles 
should be avoided unless the maker of the bibliography 
has the skill and training needed to do it consistently. 


III. Other Considerations 


Besides the questions of form and content, previ- 
ously discussed, there are certain general considera- 
tions relating to the research report which should be 
considered. These are given in the following para- 
graphs: 

1. Organizing and writing the report. The actual 
process of organizing and writing the report will natur- 
ally vary with the problem and with the individual do- 
ing the work, but the following suggestions may have 
some general applicability: (1) Careful records of all | 
steps in the study should be kept as the study 
- progresses, in order to be available when the time 
comes to write them up. (2) All materials to be used 
in composing the report should be kept in loose-leaf 
form to facilitate classification and rearrangement. 
(3) An outline should be made dbefore rather than after 
the report is written, in order to provide greater pro- 
portion and order in the whole report. (4) The job of 
writing the report should be divided into parts so that 
the author can do one at a time without having to be 
conscious of all the other parts at once. It is impos- 
sible to write the report as a whole or in general as a 
single unit. That is more than the mind can manage 
at one time. (5) A period of time should be allowed 
to elapse between the writing of the first and second 
drafts, in order to permit the mind to come back to 
its previous output with a fresh viewpoint. 

2. Audience. Whether to address a research report 
to the specialist or to the laymen is a question of con- 
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siderable importance. Unless the report is written so 
that the layman can read and understand it, it is likely 
to fail to have much effect upon the actual practice or 
behavior of the laymen. There is a great need for 
popularizing the results of research, or of “humaniz- 
ing’’* knowledge, so that all may profit by the discov- 
eries of the scientists. To write the report for laymen 
increases the size of the potential audience, since there 
are many laymen and few specialists. Also, it not in- 
frequently happens that what is written for the layman 
is also quite serviceable for the specialist, and thus 
reaches all, while the converse is seldom true—seldom 
can the laymen understand a report written principally 
for the scientist. If the report is written to the popu- 
lar reader and all technical matter is presented in foot- 
notes or in the appendix, the results is applicable to 
either group. 

But it is frequently difficult to make the report sim- 
ple and non-technical and at the same time make it 
serve the purposes of the specialist. He is interested 
in the technique, while the layman is concerned chiefly 
with results. He is interested in many of the smaller 
details, while the layman is concerned more with the 
general nature of the conclusions. He wants answers 
to profound questions which the layman does not know 
exist. Consequently, he may be quite dissatisfied with 
a research report written to the popular audience. 

Since the interests of the scientist and of the layman 
are both important factors to be considered, there are 
three possible courses for the writer of the research 
report to pursue, as follows: (1) To address it to the 
layman and put technical material in the footnotes or 


*J. Harvey Robinson: “The Humanizing of Knowledge.” The 
George H. Doran Company, New York, 1924. 
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appendix; (2). to address it to the scientist and include 
summaries of the main points in simple form for the 
popular reader; (8) to write the report to the scientist 
and then bring out a popular edition of the same report 
in a style that is suitable for the layman. 

3. Style. The style of a report will depend to some 
extent upon the audience to which it is addressed, but 
several things apply to the style for any kind of audi- 
ence. These may be mentioned as follows: (1) The re- 
port should be written up as a report upon an investiga- 
tion rather than as an essay composed by the investi- 
gator. (2) It should be written in a modest vein, 
avoiding the use of the first person pronoun, I, and 
avoiding praise of the study or boastfulness about the 
importance of the findings. (8) It should be written 
in proisaic or matter-of-fact, rather than poetic or fig- 
urative style. Glowing similes and metaphors may 
catch the popular fancy, but they often lack the ability 
to impart ideas clearly and accurately. (4) It should 
be clear, so that it actually leaves an accurate and 
definite impression in the reader’s mind. (5) It should 
be interesting. This is to be achieved more through 
relying upon the intrinsic merits of the investigation 
than by attempts at racy journalistic style or resort to 
anecdotes and facetious comments. (6) It should in- 
volve a reasonable amount of concreteness or of illus- 
trative material which helps the mind to grasp the 
significance of the abstract elements. Technical terms 
do not have as rich a fund of meaning associated with 
them in the reader’s mind as in the writer’s; there- 
fore some special effort on the writer’s part is required 
to supply that deficiency. (7) It should be free from 
the encyclopedic tendency; that is, the report should be 
more than a miscellaneous presentation of numbers of 
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isolated details. It should represent an organization 
and sequence of points which impress the reader as 
being the natural and inevitable way in which they 
should be presented. This suggests that the material 
should not be presented in long unbroken paragraphs, 
and also that it should not be presented in long series 
of paragraphs without larger heads or classifications. 

4, Seale. A common tendency of beginners in re- 
porting research is to write the report on too condensed 
a scale. While the opposite tendency, or padding, is 
sometimes met with, it is not nearly so common in the 
case of conscientious persons who are really trying to 
do their best. 

In general the scale upon which the research report 
is written should be one of considerable detail rather 
than of considerable brevity, for two main reasons: 

(1) For understanding. The report is not nearly so 
clear to the reader as to the writer. After months of 
thought upon the study, the writer can hardly realize 
that others are not so well informed about it as he is. 
He tends to write the report to others who knows as 
much about it as he does, or to his research advisor 
who has counselled with him freely, and thus it tends 
to go beyond the comprehension of his readers. For 
this reason it is well to space out the report, giving 
paragraphs to points which seem to deserve lines or 
words, and thus allowing sufficient space in which to 
bring in explanations or illustrations of points. The 
beginner will need to do for his report somewhat the 
same thing that the physical director does with his 
students before a set of exercises. That is, he must 
“open ranks” and give room for each individual idea 
to develop itself. Possibly one of the most helpful 
pieces of advice to be given to thesis writers in regard 
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to this point is the injunction, “Don’t crowd.” There is 
room for everything in the report if it deserves a place, 
as there is seldom a page limitation (except in cases of 
publication) and there is no reason for crowding much 
into a little space. 

(2) The second reason for a full and detailed report 
is the desirability of preserving a record of everything 
of any significance. A research report resembles a 
court record in that it is a depository for every fact or 
piece of information that could possibly have any sig- 
nificance, or that might need to be consulted at a later 
time. It is to be used later more for reference than 
for connected reading for pleasure, and for that reason 
should contain such information as may be desired for 
later reference. It should contain a sufficiently com- 
plete record of the procedure that the reader cannot 
only judge for himself whether the conclusions are 
warranted, but also that he could set up and carry out 
a second study identical to it for purposes of verifica- 
tion of the findings. It should not only report the find- 
ings in enough detail to lend conviction as to their 
soundness, but also include the original data, scores, or 
raw materials needed by anyone who wishes to check 
the calculations or to tabulate and treat the data by 
other supplementary procedures. If the report is to be 
of the maximum value to the cause of science, it should 
contain enough information to make it unnecessary for 
the writer to supply supplementary details from his 
own memory or from personal records. 

An exception to all this exists in some cases in pre- 
paring reports for publication where the expense or 
available page allowance necessitates condensation. 
But even in such cases the full and detailed record 
should be made, and the published report should refer 
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to it in a way that will enable any interested reader 
to learn where he can consult it. 

5. Publication. Not many research reports are of 
sufficient general interest to warrant publication in 
complete form, but most investigations that are well 
done should warrant publication at least as articles or 
summaries. Several suggestions may be made: (1) It 
is well not to rush into print without some actual sub- 
stance to report, and not to publish half-baked or ill- 
considered findings. (2) Only one main idea or find- 
ing should be presented in an article, particularly for 
the popular reader. If many findings are to be re- 
ported, many articles may be published, each on a dif- 
ferent phase. Care, however, should be exercised not 
to multiply articles unnecessarily. (3) Cooperation of 
the research adviser in securing publication is desir- 
able. He may either edit the article before it igs sent 
off, or may submit it to a journal with his reeommenda- 
tion. Or he may even write the article from the full 
report and publish it jointly with the original investi- 
gator. (4) Chances of publication may be materially 
improved if the article is planned so as to require small 
space and low printing cost. Elaborate tables or ex- 
pensive charts and graphs may increase the cost to a 
prohibitive degree. (5) The chances of publication are 
increased by seeking the appropriate publisher. Jour- 
nals specialize in different fields, and what one would 
like another could not use. Articles that would not 
interest national journals might appeal to local or state 
journals, and vice versa. Much can be done also by 
writing the article to suit the type of readers who sub- 
scribe to the journal in which publication ig sought. 
(6) It may be better in some instances to let another 
person write the article for publication, as in the case 
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where a journalist is called in to dress up in popular 
style the results of an investigation of a very abstract 
nature. 

6. English. It is needless to say that a research re- 
port should be written in good English. How to achieve 
that end, however, deserves more attention. Three sug- 
gestions may be in order, as follows: (1) The writer 
of the report should consider the “architecture” of his 
English as well as the “carpentry” of it—that is, he 
should look to the soundness of the larger plan of or- 
ganization of his report as well as to the correctness 
of the details of word usage. (2) He might well afford 
to purchase and use for reference some standard hand- 
book on the mechanics of English, or some standard 
manual of style. (38) He should ordinarily have some- 
one else read his report after him to correct such errors 
of English as he has missed. If he writes a very bad 
quality of English, he may have to employ someone to 
make a detailed study and to revise his report in regard 
to its English form. 

In closing, we may say that the good research report 
is one which clearly describes what was done and what 
was found, and does this so effectively that busy read- 
ers will take time to read, think, and act upon the 
findings. 


EXERCISES AND QUESTIONS 


1. Give every reason you can for writing up the results in a 
full and complete report. 


2. In what ways should a report that is to be published as a 
brief article differ from the more extensive unpublished 
report? Why? 


3. Explain why most research is “buried” on completion, never 
being used by those whom it might possibly serve. How 
could this situation be remedied? 
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4, Examine a few research reports and criticize their organi- 
zation, style, and content. 


5. To what extent would it be possible or desirable to fit every 
research report into the same general set of chapter 
titles? Why? 


6. Give every method you can for popularizing research find- 
ing'’s. 


7. Compare the booklets of directions regarding thesis form 
from several graduate schools with the specifications in 
this chapter and see how well they agree on the vari- 
ous points. 


8. Explain why theses in different departments in a graduate 
school vary so greatly in content and organization. 
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